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The  course  in  which  you  are  about  to  participate,  entitled  "Introduction  to 
Program  Management."  and  this  accompanying  handbook  are  intended  to  provide 
you  with  an  awareness  of  the  scope  and  importance  of  program  management  at 
the  Naval  Ocean  Systems  Center  and  to  impart  to  you  the  management 
philosophies  that  I  believe  will  be  important  to  your  success,  individual 
development,  and  career  growth. 

It  is  my  belief  that  NOSC  is  a  key  part  of  the  Navy's  Systems  Acquisition  Team 
and  that  our  Center  s  reputation  depends  on  the  capabilities  of  our  program 
managers  Therefore,  it  is  imperative  that  each  individual  charged  with  program 
management  responsibilities  discharges  those  responsibilities  in  a  way  that  fulfills 
DoD  and  Navy  statutory  requirements  and  ensures  that  the  Navy  receives  the 
maximum  benefit  from  its  investment.  In  addition  to  these  legalistic  requirements, 
we  must  operate  with  efficiency  and  complete  integrity  And.  while  it  is  a  well 
worn  phrase,  it  really  is  often  our  responsibility  to  ensure  that  the  Navy  is  able  to 
operate  as  a  smart  buyer  in  a  very  complicated  and  highly  technical  market 
place  In  the  final  analysis  we  have  succeeded  only  if  we  provide  the  Navy  with 
effective  and  affordable  timely  solutions  and  capabilities  that  are  superior  to  those 
of  the  threat 

It  is  my  intent  that  this  course  provide  the  framework  for  training  new  and 
potential  program  managers  as  well  as  establishing  a  standard  by  which 
functioning  program  managers  can  measure  themselves  Because  this  course  and 
handbook  are  directed  toward  program  management  as  it  is  accomplished  at  the 
Naval  Ocean  Systems  Center,  your  feedback  on  the  course  and  supporting 
documentation  is  encouraged  As  the  need  arises,  this  course  and  document  will 
be  updated  to  reflect  changes  in  the  program  management  environment  as  well  as 
changes  in  statutory  law  governing  the  methods  of  accomplishing  our  business. 
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SECTION  1 

EXECUTIVE  SUMMARY 
R.  Watts,  Code  942 

1.1  PURPOSE 

The  NOSC  Program  Managers'  Handbook  is  designed  as  a  ready  reference  document  for  those 
who  have  taken  the  NOSC  course,  “Introduction  to  Program  Management.”  It  is  not  intended  to  be 
a  thorough  examination  of  each  topic.  However,  the  handbook  provides  its  user  with  basic  information 
regarding  the  topic  along  with  some  references  and  resources,  including  NOSC  instructions  and  other 
applicable  documentation.  It  also  provides  guidance  for  program  managers*  as  they  carry  out  their 
varied  functions,  and  gives  examples  of  the  forms  or  requirements  that  accompany  particular  activities. 

The  target  audience  for  this  handbook  is  a  group  of  interested,  skilled,  and  experienced  program 
managers  who  know  what  it  is  like  to  manage  a  program  in  the  real  world.  Perhaps  the  thought  has 
occurred  to  them  that  the  “system”  does  not  work.  Perhaps  their  experience  has  reflected  the  following 
six  phases  of  a  project: 

Enthusiasm 

Disillusionment 

Panic 

Search  for  the  Guilty 

Punishment  of  the  Innocent 

Praise  and  Honors  for  the  Nonparticipants. 

However,  part  of  the  purpose  of  this  handbook  is  to  demonstrate  that  the  “system”  does  work  when 
implemented  by  skilled  program  managers.  Center  management  provides  policy  and  guidance  certainly, 
but  it  is  the  task  and  responsibility  of  program  managers  to  implement  the  guidance  and  policy,  i.e. 
to  make  the  “system”  work.  It  is  hoped  that  the  users  of  the  handbook  will  be  those  successful  managers. 


1.2  ORGANIZATION 

The  NOSC  Program  Managers'  Handbook  contains  an  executive  summary  and  sections  for  each 
of  the  18  topics  addressed  during  the  management  course: 

Section  1 .  Executive  Summary 

Section  2.  How  Projects  Originate  and  Develop 

Section  3.  Program  Management  Functions  and  Responsibilities 

Section  4.  Proposal  Development  and  Marketing 

Section  5.  Staffing,  Team  Building,  and  Communication 

Section  6.  Computer  Tools  for  Program  Managers 

Section  7.  Planning,  Scheduling,  and  Assessment 

•It  it  realized  that  “program  managers"  hat  a  particular  DoD  denotation.  However,  in  this  handbook  the  term  is  used  in 
a  broad  sense  and  includes  program,  project,  and  product  managers. 
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Section  8. 
Section  9. 
Section  10. 
Section  11. 
Section  12. 
Section  13. 
Section  14. 
Section  15. 
Section  16. 
Section  17. 
Section  18. 
Section  19. 


FORMAT 


Systems  Engineering 
Major  Systems  Acquisition 
Technical  Information  Support 
Hardware  Product  Assurance 
Test  and  Evaluation 
Computer-Aided  Logistics 
Software  Product  Assurance 
Contracting 

Budget  and  Financial  Management 
Design  Review 
Follow-on  Training 
Human  Factors. 


Apart  from  the  executive  summary,  each  of  the  sections  follows  a  general,  numbered  outline.  That 
numbered  outline  depends  first  on  the  section  number  (2  through  19).  The  first  subsection  (X.l)  is 
always  the  introduction,  and  it  has  three  subsections:  references,  outline,  and  summary.  From  that 
point  on,  the  outline  is  unique  for  each  section. 

1.4  AN  INVITATION 

This  handbook  is  a  document  in  process.  As  such  it  will  be  constantly  updated  and  revised  to 
reflect  the  latest  information  and  thought  in  each  of  the  topic  areas.  If  any  contributor  wishes  to  revise 
and  extend  his  remarks,  he  is  invited  to  do  so.  If  any  of  the  participants  has  suggestions  on  how  to 
improve  the  handbook  or  ideas  on  how  the  material  could  be  more  effectively  presented,  please  contact 
the  cognizant  personnel.  Any  constructive  counsel  is  welcome. 


1.5  ACKNOWLEDGMENTS 

Grateful  thanks  are  extended  to  the  session  leaders  for  the  time  and  effort  expended  in  preparing 
their  presentations.  Each  one  of  them  is  a  volunteer  who  thought  that  the  topic  was  significant,  that 
it  deserved  airing  in  front  of  Center  program  managers,  and  that  NOSC  would  perform  its  mission 
better  with  better  informed  program  managers. 
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SECTION  2 

HOW  PROJECTS  ORIGINATE  AND  DEVELOP 
D.  Washburn,  Code  7403 


2.1  INTRODUCTION 


2.1.1  References 

Navy  Program  Manager’s  Guide,  1985 
Navy  RDT&E  Management  Guide,  NAVSOP-245I 
NOSC  Planning  Calendar,  NOSC  TD  847 
NOSC  Program  Planning  Guidance  Memorandum 

The  Planning,  Programming,  and  Budgeting  System  (PPBS)  Course  Materials,  Office  of  the 
Director,  DON  Program  Information  Center  (26  June  1985),  Washington,  DC  20350. 

DoDI  5000.2  and  DoDI  5000.3 
SECNAVINST.  5000.1 
OPNAVINST  5000.42 

NOSC  Memo  013/49-85,  Policy  and  Procedures  for  FY86  IR  Program,  of  20  March  1985. 
NOSC  Memo  014/173-85,  FY86  IED  Proposals,  of  9  August  1985 

NOSC  Memo  021/21-85,  Information  on  Major  Bid  and  Proposal  (MB&P)  Program,  of  18 
September  1985. 


2.1.2  Outline 

Introduction 

References 

Outline 

Summary 

General 


2.1.3  Summary 

See  below. 


2.2  GENERAL 

The  conduct  of  research  and  development  is  focused  upon  meeting  Fleet  needs  in  the  near  term 
and  distant  future.  Because  resources  are  limited  and  needs  can  arise  with  alarming  suddenness,  careful 
and  responsive  program  origination  and  management  are  required.  This  class  session  will  offer  a  brief 


overview  of  the  existing  organizations  and  processes  employed  in  Navy  research,  development,  test, 
and  evaluation. 

While  Fleet  needs  and  deficiencies  are  topics  of  semiannual  messages  to  and  from  Fleet  commanders, 
they  are  the  constant  topics  of  discussion  throughout  the  Navy,  Marine  Corps,  and  Coast  Guard.  Arising 
threats,  new  mission  roles,  obsolescence  of  existing  capabilities  and  equipment,  technology  breakthroughs 
and  maturation,  and  the  needs  posed  by  joint-force  operations  and  foreign-sales  demands  all  affect 
initiation,  scope,  and  direction  of  research  and  development  efforts. 

In  our  age  of  technology  where  so  much  can  be  done,  the  matter  of  choice  in  applications  engineering 
is  crucial  to  national  defense.  The  utility  of  ancient  simple  weapons  was  extensive  and  long-lived.  Modern 
weapons  systems  are  tailormade  to  exploit  a  diminishing  set  of  target  vulnerabilities  and  countermeasures, 
and  so  are  more  expensive  to  field  and  of  shorter  utility  against  a  responsive  threat  organization. 

In  efforts  to  respond  to  changes  in  world  and  national  circumstances,  maritime  strategies  and  U.S. 
Navy  roles  and  objectives  can  also  change  in  function,  scope,  and  priority.  National  requirements  for 
purposes  of  “presence,”  “sea  control,”  and  “projections  of  force,”  etc.  shift  in  emphasis.  Balance 
of  appropriate  roles  and  strengths  among  our  armed  forces  also  affects  R&D  objectives,  resources, 
and  the  economics  and  utility  of  logistics. 

Timing  is  as  essential  in  RDT&E  as  it  is  in  the  capability  and  technology  choices;  seldom  is  cost 
not  a  governor  of  the  process.  Because  resource  needs  normally  exceed  resources,  the  Navy  employs 
an  advocacy  system  in  OPNAV  and  between  the  services  in  DoD  to  hammer  out  the  Five-Year  Defense 
Program  (FYDP).  The  unfortunate  feature  of  the  process  is  that  there  is  the  need  for  an  almost  constant, 
urgent  defense  of  proposals  against  often  worthy  alternatives,  all  of  which  may  be  valid  and  timely 
options. 

The  material  for  this  overview  of  RDT&E  program  management  has  been  drawn  largely  from 
two  documents: 

a.  The  Navy  Program  Manager’s  Guide,  1985  edition  NAVMAT  P-9494  (NOSC  Library  Accession 
1 10582) 

b.  The  Planning,  Programming,  and  Budgeting  System  (PPBS)  course  materials,  Office  of  the 
Director,  DON  Program  Information  Center  (26  June  1985),  Washington,  DC  20350. 

Each  document  refers  the  reader  to  an  extensive  list  of  documents  in  order  to  provide  current 
authoritative  information  addressing  specifics.  The  grand  scheme,  however,  is  apparent  in  a  review 
of  the  major  concepts  and  processes  of  the  organizations  presented  in  these  recent  documents,  each 
of  which  is  subject  to  revision  at  least  annually.  The  Navy  Program  Manager’s  Guide,  for  instance, 
is  in  revision  in  part  because  NAVMAT  no  longer  exists.  The  U.S.  Navy  Postgraduate  School  (NPGS) 
is  the  new  sponsor  for  this  document.  You  will  find  this  source  very  readable  and  useful,  and  so  should 
consider  it  as  a  desk  reference  for  your  program  management  office. 

As  stated  in  the  introduction  to  the  Navy  Program  Manager's  Guide,  the  purpose  is  to  assist  the 
manager  by  outlining  the  system  acquisition  process,  identifying  participants  and  describing  their  roles, 
describing  the  procedures  necessary  to  move  the  program  from  one  milestone  to  the  next,  and  identifying 
possible  pitfalls  along  the  way.  This  guide  was  prepared  under  direction  of  Dr.  George  Handler  of 
the  Naval  Weapons  Center,  China  Lake,  California.  Because  it  is  so  extensive  and  provides  details 
useful  to  program  managers  in  Navy  laboratories,  as  well  as  in  headquarters  organizations,  it  is  an 
important  text  to  accompany  the  presentation  for  our  topic:  How  Projects  Originate  and  Develop. 


NOSC  program  management  guidance  and  support  are  described  in  an  extensive  series  of  NOSC 
instructions,  directives,  and  notes  kept  current  by  the  various  offices  responsible.  Keeping  current  on 
all  of  these  is  difficult  without  maintaining  a  set  of  current  instructions  in  the  program  office  for  reference. 
Indeed,  setting  up  a  program  office  is  left  to  the  ingenuity  of  the  manager,  though  organization,  files, 
and  operations  should  be  described  in  documents  maintained  for  convenience  and  efficiency  among 
employees  of  the  office. 

NOSC  memorandums  describing  organization  and  independent  research  (IR),  independent 
exploratory  development  (IED),  and  bid  and  proposal  processes  include  the  following: 

NOSC  Memo  013/49-85,  Policy  and  Procedures  for  FY86  IR  Program,  of  20  March  1985. 

NOSC  Memo  014/173-85,  FY86  IED  Proposals,  of  9  August  1985. 

NOSC  Memo  021/21-85,  Information  on  Major  Bid  and  Proposal  (MB&P)  Program,  of  18 

September  1985. 

A  most  useful  NOSC  document,  TD  847,  is  a  single  page,  poster-size  schedule  summary  for  R&D 
planning  guidance.  The  title  is  NOSC  Planning  Calendar ,  and  it  will  be  updated  annually. 

It  should  be  noted  that  the  involvement  of  NOSC  in  more  responsible  roles  in  major  program 
management  makes  it  desirable  for  aspiring  managers  to  attend  one  or  more  of  the  following  more 
extensive  courses  on  the  subject: 

Defense  Systems  Management  College 

Naval  Material  Career  Development  Institute 

Federal  Acquisition  Institute 

USN  or  Department  of  Defense  PPBS  course  (joint  service  programs  are  also  addressed). 

The  most  important  documents  shaping  current  acquisition  policy  are  Department  of  Defense  Doc. 
5000.1  and  its  implementing  instructions: 

DoDI  5000.2  and  DoDI  5000.3 

SECNAV  INST.  5000.1 

OPNAV  INST  5000.42. 

These  are  drafted  addressing  major  programs,  for  which  the  Secretary  of  Defense  chooses  to  act  as 
program  decision  authority  (PDA),  but  each  applies  to  all  lesser  programs  in  principle,  when  tailored 
to  the  nature  and  cost  of  each. 

Generally,  DON  acquisition  policy  calls  for  a  program  initiation  decision  to  be  made  by  the  proper 
program  decision  authority  and  approval  for  program  start  to  be  integrated  with  the  planning, 
programming,  and  budgeting  system  (PPBS).  At  each  subsequent  major  milestone,  the  program  manager 
is  required  to  prepare  milestone  review  documentation  (MRD)  and  have  it  reviewed  and  submitted  to 
the  PDA  for  approval. 

NOSC  program  or  project  managers  are  required  to  provide  documentation  support  for  sponsor 
compliance  with  these  requirements,  together  with  representation  of  technical  work  and  planning. 
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SECTION  3 

PROGRAM  MANAGEMENT  FUNCTIONS  AND  RESPONSIBILITIES— 
RELATIONSHIPS  TO  LINE  MANAGEMENT 
D.  Washburn,  Code  7403 

3.1  INTRODUCTION 


3.1.1  References 

Navy  Program  Management  Guide,  1985 
NOSC  Program  Planning  Guidance  Memorandum 
NOSC  Instructions  &  Notes 

3.1.2  Outline 

Program  Establishment  Steps  at  NOSC 
Program  Manager  Charter  and  Plan 
Lab  Center  Functions  in  RDT&E 
Acquisition  Improvement  Initiatives 
External  Influences 
Organizational  Effectiveness  Review 
R&D  Productivity 
Ethics 

3.1.3  Summary 

See  3.2  below. 


3.2  GENERAL 

The  establishment,  conduct,  and  conclusion  of  an  RDT&E  program  at  NOSC  presumes  awareness 
and  compliance  with  basic  functions,  forms,  and  communication  standards  imposed  by  the  sponsor 
and  by  local  and  higher  organizational  levels  of  authority  and  resource  allocation.  The  NOSC  independent 
exploratory  development  (IED)  and  technical  base  management  organization  is  shown  in  Figure  3.1. 

Above  all,  the  program  must  fit  in  with  other  efforts.  In  order  to  maintain  support  for  the  program, 
the  sponsor  must  be  given  adequate  results  in  each  component  area.  Most  R&D  programs  are  composites 
of  efforts  at  several  sites  among  industry,  academic,  and  Navy  activities. 

It  is  crucial  that  the  various  program  participants  set  and  meet  milestones  and  represent  progress 
and  products  accurately  for  the  program’s  sponsor.  This  support  will  enable  your  sponsor  to  deliver 
on  his  obligations  successfully. 


In  order  to  conduct  your  program  at  NOSC,  your  program  must  fit  in  with  local  resources  and 
services;  this  includes  your  use  of  contractual  assistance. 

There  will  be  presentations  describing  most  of  the  NOSC  support  capabilities  available  during 
this  course.  The  purpose  of  this  section  is  to  describe  the  relationship  of  the  program  manager  to  other 
NOSC  organizational  groups  and  to  outside  organizations.  Organizational  details  are  subject  to  change, 
but  the  essentials  remain  valid. 

The  role  of  the  program  manager  is  not  that  of  a  big  or  bigger  “wheel”  in  the  process,  but  rather 
a  combination  of  initiator,  catalyst,  counselor,  investigator,  and  reporter,  among  others.  Responsibility, 
like  virtue,  is  its  own  reward,  but  it  is  essential  in  RDT&E  programs.  A  prime  example  is  in  the  NASA 
space  shuttle  program,  where  “success  breeds  failure,”  just  as  it  does  in  large,  challenging,  successful 
Navy  programs.  We  are  tempted  to  take  each  further  extension  or  application  for  granted.  It  is  the 
responsibility  of  the  program  manager  to  examine  and  test  the  waters  before  taking  risks  and  venturing 
outside  of  the  planned  process  and  schedule,  even  though  there  are  demands  to  change  at  every  step. 

The  Navy,  as  part  of  DoD,  is  party  to  joint  program  developments  as  well  as  independent  efforts, 
so  oversight  by  higher  DoD  managers  has  resulted  in  the  adoption  of  24  initiatives  to  improve  the 
acquisition  process  (Table  3.1).  These  initiatives  recognize  potential  and  existent  frailties  in  our  RDT&E 
programs,  most  which  recur  and  must  receive  corrective  attention  or  prevention  through  constant 
awareness. 

During  the  various  stages  of  RDT&E,  external  influences  affect  decisions  and  progress.  Perhaps 
the  ultimate  influence  is  competition  for  limited  resources,  which  can  happen  at  any  stage  and  any 
organizational  level.  Priority  is  necessary  to  compete  successfully  for  resources.  Anticipation,  planning, 
and  negotiation  will  normally  avoid  confrontation.  Conflict  is  costly.  Paving  the  way  well  ahead  of 
program  needs  is  effort  well  spent. 

The  complexity  of  doing  RDT&E  business  increases  daily.  More  plans,  reports,  status-keeping, 
and  audits  are  to  be  expected.  Only  good  organization  and  procedures  maintained  throughout  the 
program  can  satisfy  the  demands,  while  freeing  the  productive  personnel  to  conduct  the  program. 

Team  development  is  an  important  topic  which  will  be  stressed  later  in  the  course.  The  essential 
requirements  for  development  toward  future  programs  can  be  stressed  here.  The  justification  for  this 
course  has  existed  for  many  years.  Program  managers  have  been  developed  in  all  the  different  ways 
up  to  now.  Some  came  with  industrial  experience,  some  with  academic  training,  and  most  have  developed 
through  on-the-job  training  with  a  mentor  at  some  stage  in  their  careers.  All  were  shaped  by  the  teams 
of  which  they  were  a  part. 

The  performance  of  a  project  team  must  be  reviewed  by  time  management  and  their  program 
progress  and  success  must  be  monitored  as  well.  Quality  assurance  in  individual  employee  performance 
is  the  objective  of  the  demonstration  program,  which  relies  upon  performance  toward  objectives,  with 
its  incentive  awards.  No  less  important  are  assessment  and  incentive  awards  for  program  teams  and 
managers  for  the  quality  of  their  performance  and  products  measured  against  system  performance  goals 
and  schedule  and  cost  targets. 

Every  citizen  has  the  right  to  expect  ethical  behavior  from  those  who  work  in  government.  The 
principles  are  clear,  and  corruption  is  widely  reported  and  punished.  Ethics,  however,  are  applied  in 
the  subtle  everyday  actions  and  relationships  within  your  team.  Fairness  and  compliance  are  perhaps 
useful  watchwords. 


Tabic  3.1.  DoD  acquisition  improvement  program — the  Carlucci  Initiatives  (Kuhi  revision) 

Program  managers  shall  .  .  . 

1.  Be  given  responsibility,  authority,  resources,  and  proper  requirements  and  funding  statements. 

2.  Be  given  authority  to  be  flexible  in  tailoring  acquisition  strategy. 

3.  Extend  responsibility,  authority,  and  accountability  to  the  lowest  effective  organizational  level. 

4.  Examine  low  and  high  risk  technologies  in  acquisition  strategy  development. 

5.  Consider  program  improvement  in  program  planning. 

6.  Pursue  economical  rates  of  production  within  constraints  as  a  basic  goal. 

7.  Consider  and  pursue  a  policy  of  standardization  whenever  and  wherever  beneficial. 

8.  Ensure  that  DON  personnel  take  a  businesslike  approach  with  industry  in  terms  of  motivation 
and  teamwork  goals. 

9.  Solicit  industry’s  comments  on  draft  PFPs  when  those  comments  are  likely  to  be  beneficial. 

10.  Inform  industry  accurately  regarding  the  funding  available  for  a  particular  program  and  not  mislead 
in  any  way. 

11.  Ensure  that  acquisition  strategies  ascribe  value  to  a  viable  industrial  base. 

12.  Procure  data  only  when  needed  and  if  it  is  sufficient  for  life-cycle  maintenance. 

13.  Provide  effective  estimates  of  resources  and  see  that  they  are  used  throughout. 

14.  Use  realistic  estimates  for  program  budgets  and  schedule  profiles. 

15.  Minimize  total  life-cycle  costs  with  a  view  to  influence  acquisition  strategy. 

16.  Emphasize  value  engineering  when  participating  in  cost  saving  programs. 

17.  Consider  multiyear  procurement  in  all  applicable  situations. 

18.  Employ  independent  Navy  cost  analysis  in  contract  negotiations  whenever  possible. 

19.  Pursue  competition  vigorously  when  a  potential  benefit  exists. 

20.  Minimize  contract  changes;  but  once  changes  have  been  issued,  expedite  them. 

21.  Expedite  the  entire  acquisition  process  to  the  greatest  degree  possible. 

22.  Use  past  performance,  experience,  and  cost  realism  in  source  selection  and  cost  reimbursable 
contracting. 

23.  Emphasize  reliability,  maintainability,  and  produceability  from  initial  design  onward. 

24.  Insist  that  logistic  support  standards  will  not  be  compromised. 
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Being  overly  accommodating  can  be  ruinous.  Security  breaches  are  often  traced  to  accommodation: 
too  little  time  to  check  the  area,  to  check  the  lock,  to  log  documents,  or  to  safeguard  the  information 
from  unauthorized  disclosures.  Overly  accommodating  poor  performance  by  members  places  a  weak 
link  in  your  chain  and  undue  burdens  on  other  team  members — who  notice  and  respond.  Accommodating 
imposed  changes  in  schedule  or  system  details — without  assessing  impact  downstream  on  other 
obligations — can  wreck  the  best  program  through  loss  of  performance,  credibility,  sponsorship,  and 
future  opportunity. 

The  references  offered  above  are  doctrinal  and  procedural.  The  essence  of  how  to  do  program 
management,  with  enjoyment,  comes  from  reading,  discussing,  and  experiencing  management  functions. 
I  hope  that  this  portion  of  this  course  makes  the  role  appear  attractive,  challenging,  and  rewarding. 

It  is  said  that  the  best  recipe  for  stress  avoidance  is: 

Expect  changes 

Assist  others  to  expect  and  accommodate  changes 

Reward  yourself  for  things  you  do  well  and  try  to  make  that  your  style. 

As  Helen  Hayes  responded  to  an  interviewer: 

Success  is  how  others  assess  what  you  have  done; 

achievement  is  your  own  assessment. 

Figures  3.2  through  3.6  present  general  information  for  program  management  and  cover  such  things 
as  the  needs  document;  the  planning,  programming,  and  budgeting  system  (PPBS)  events;  and  the 
acquisition  process. 
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Figure  3.3.  Sequence  of  PPBS  events. 
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Figure  3.4.  Summary  overview  of  the  acquisition  process. 
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Figure  3.5.  Acquisition  phases  milestones 
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Figure  3.6.  Interfaces  of  events,  documents,  and  organizations  in  the  acquisition  process. 
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SECTION  4 

PROPOSAL  DEVELOPMENT  AND  MARKETING 
E.  Towson 


4.1  INTRODUCTION 


4.1.1  References 

Navy  Program  Managers  Guide 

Navy  RDT&E  Information,  Navy  POM-87,  FY  1987-91,  May  1985  (SECRET) 

Program  Objective  Memorandum,  Volume  Five:  Extended  Planning  Annex  (EPA),  FY 
1987-1991  (POM-67)  (SECRET) 

NAVSEA  Technology  Needs  Guidance,  Volume  1,  prepared  by  Research  and  Technology 
Office,  Naval  Sea  Systems  Command  (SECRET  WORKING  PAPERS) 

Joint  Strategic  Planning  Document  (JSPD),  The  Joint  Chiefs  of  Staff,  4  September  1984 
(SECRET) 

Surface  Ship  Combat  System  Master  Plan  (U),  Volume  1,  published  by  direction  of 
Commander,  Naval  Sea  Systems  Command,  October  1985  (SECRET  —  NOFORN) 
Navy  Command  and  Control  Plan  (U),  Chief  of  Naval  Operations  (SECRET) 

NOSC  Olympus  Updates,  prepared  by  Earl  Towson,  January  1986  (SECRET) 

Laboratory  Program  Summary  (1498s)* 

NOSC  TD  799,  Windows  of  Opportunity  for  Naval  Systems 

•This  is  THE  source  document 


4.1.2  Outline 

Introduction 
References 
Outline 
Summary 
Finding  a  Sponsor 
Market  Research 
The  System  Acquisition  Process 
Developing  Proposal 
New  Marketing  Strategies 
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4.1.3  Summary 


In  summary,  you  as  the  good  program  manager,  when  involved  in  proposal  development  and 
marketing,  should  take  the  following  steps: 

a.  Conduct  market  research  yourself.  Ask  yourself  the  relevant  questions.  What  kind  of  business 
am  I  in?  What  is  my  product  area?  Product  line?  What  am  I  good  at?  Where  am  I  weak? 
How  do  I  use  the  team  conception  to  overcome  the  weakness?  How  can  I  take  full  advantage 
of  my  strengths.  These  questions  can  be  addressed  in  a  number  of  ways.  First,  do  your 
homework.  Second,  conduct  interviews  (this  is  a  key  element).  Third,  use  the  help  available 
in  local  offices  (Code  163,  the  Technical  Library,  NARDIC  (Pasadena),  etc.).  Finally,  develop 
client/advisor  relationships. 

b.  Pick  your  targets  carefully.  You  should  match  your  strengths  and  put  together  a  team  to  fill 
the  voids. 

c.  Know  the  planning,  programming,  and  budgeting  system  (PPBS)  and  the  system  acquisition 
process.  Besides  knowing  the  system  and  the  process,  you  should  also  know  the  players  and 
when  to  play  (and  when  to  hold,  fold,  and  run!) 

d.  Develop  winning  proposals.  You  should  make  strong  presentations  (do  not  depend  on  the 
strength  of  your  idea  or  design).  Also  it  is  necessary  to  be  aware  of  the  calendar:  feed  the 
“paper  tiger”  on  time,  get  documents,  reports,  and  forms  in  on  time.  In  addition,  watch  the 
competition,  be  aware  of  what  is  going  on  in  your  area  of  interest. 

And  finally  some  advice  for  future  winners: 

a.  Sell!  Sell!  Sell! 

b.  Don  ’t  play  the  game  unless  you  can  win. 

c.  Don't  slug  it  out  over  objects  of  questionable  value;  however,  if  something  is  wrong  say  so. 

d.  Don’t  stay  with  a  loser  —  this  is  the  cardinal  sin. 

e.  Don’t  juggle  too  many  balls  at  once,  too  many  programs  at  once. 

f.  Anticipate  OSD,  OMB,  etc.  issues  and  help  service  reclamas. 

g.  Conduct  institutional  advertising  (e.g.  why  Navy  labs?  why  more  defense  dollars?) 

h.  Maintain  an  awareness  of  the  competition.  Know  what  they  are  doing  and  what  they  are 
planning  to  do. 

i.  Promote  the  corporate  image  as  well  as  the  product. 


4.2  FINDING  A  SPONSOR 

Do  your  homework.  Review  and  study  the  appropriate  referenced  documents.  There  are  many 
very  helpful  documents  that  are  seldom  used.  There  are  budget,  guidance,  and  planning  documents 
available.  For  example.  Code  16  is  collecting  documentation  in  these  areas;  Table  4.1  lists  some  of 
the  major  planning  documents  that  are  available.  So  study  the  basic  documents  that  have  to  do  with 
your  area  of  interest.  Keep  in  mind  the  following  questions.  What  does  the  Navy  need?  Where  is  the 
threat  going?  How  much  money  is  flowing  in  my  business?  Who  is  buying?  Selling?  You  should  know 


Table  4.1.  Major  planning  documents. 


ASW  Master  Plan 

Attack  Sub  Warfare  Plan 

EW  Master  Plan 

Extended  Planning  Annex 

USMC  Mid-Range  Objectives  Plan 

Master  Navigation  Plan 

Master  Plan  for  Embedded  Computers 

Mine  Warfare  Master  Plan 

Avionics  Master  Plan 

Naval  Aviation  Plan 

Naval  C2  Plan 


Special  Warfare  Master  Plan 
Combat  Systems  Master  Plan 
Surface  Warfare  Plan 
USMC  C2  Master  Plan 
NORAD  Master  Plan  (USAF) 

Ocean  Surveillance  Master  Plan 
Space  Master  Plan 
Avionics  Planning  Baseline  (USAF) 
Strategic  Defense  Arch  2000  (USAF) 
AF  SDF  C2  Arch 
Etc.,  etc.,  etc. 


the  documents  that  apply  to  your  area  of  interest  as  well  as  your  sponsor  or  prospective  sponsor,  because 
you  are  coming  to  him  as  the  expert  who  is  going  to  solve  his  problems.  Thus,  he  is  going  to  judge 
you  and  your  credibility  as  an  expert  on  how  well  you  use  the  language  of  the  discipline  under 
consideration  and  on  the  depth  of  your  awareness  and  understanding  of  his  problems. 

In  some  cases  you  will  have  a  solution  and  will  need  to  Find  a  sponsor.  Be  on  the  lookout  for 
sponsors.  Take  advantage  of  conferences  (this  is  especially  important  for  NOSC  because  we  cannot 
go  out  and  market  like  industry  does).  Use  your  opportunity  at  conferences  to  question  the  key  players 
in  your  area  of  interest.  Also,  refer  to  the  conference  when  asking  for  appointments  in  the  future. 

What  you  face  when  approaching  a  prospective  sponsor  is  a  series  of  concerns  he  may  have  about  you: 

I  don’t  know  who  you  are. 

I  don’t  know  your  company. 

I  don’t  know  your  company’s  product. 

I  don’t  know  what  your  company  stands  for. 

I  don’t  know  your  company’s  customers. 

I  don’t  know  your  company’s  record. 

I  don’t  know  your  company’s  reputation. 

Now  —  what  was  it  you  wanted  to  sell  me? 

Your  task  then,  when  you  meet  the  prospective  sponsor,  is  that  you  must  first  sell  yourself,  then 
your  idea,  and  finally  your  organization. 
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4.3  MARKET  RESEARCH 

NOSC  TD  799,  Windows  of  Opportunity  for  Naval  Systems,  illustrates  one  method  for  discovering 
an  opportunity.  Figure  4. 1  shows  windows  of  opportunity  as  related  to  initial  operational  capability 
(IOC)  dates  for  various  representative  systems.  Examine  the  systems  in  your  area  of  interest  and  the 
necessary  questions.  When  was  the  system  bought?  When  will  it  wear  out?  What  happened  with  past 
systems?  What  is  the  status  of  current  systems?  What  systems  are  being  considered  for  the  future? 
When  will  the  window  of  opportunity  open  for  my  system? 

Another  method  for  predicting  a  possible  opportunity  involves  affordability  forecasting  (Figure 
4.2).  In  this  approach  you  examine  an  area  of  interest  such  as  ocean  surveillance  (OS),  anti-submarine 
warfare  (ASW).  command  and  control  (CC),  etc.  in  the  light  of  the  funding  history.  In  other  words 
you  are  asking,  when  does  the  cash  flow  in  my  business?  You  then  attempt  to  budget  your  system 
for  the  time  of  optimum  cash  flow  in  your  area  of  interest. 

In  subsection  4.2  it  was  noted  that  when  you  approach  a  prospective  sponsor,  you  must  first  sell 
yourself,  then  your  idea  (project),  and  finally  your  organization.  What  follows  are  suggestions  for  a 
successful  interview  with  a  prospective  sponsor: 

a.  Have  “objective"  market  analysts  do  the  interview.  (This  interview  is  done  for  information, 
not  sales,  so  you  want  to  hear  what  the  sponsor  actually  says,  not  what  you  want  to  hear. 
You  want  to  find  out  what  this  prospective  sponsor  thinks.) 

b.  Call  him  personally.  Tell  him  what  you  really  want  (again  don’t  give  him  a  sales  job  if  you 
want  an  interview). 

c.  Do  your  homework.  For  instance,  read  the  publications  authored  by  that  office. 

d.  Make  sure  your  clearance  gets  to  the  proper  locations. 

e.  Compile  a  questionnaire  ahead  of  time.  It  also  helps  to  establish  priority  among  the  questions 
in  case  the  time  for  the  interview  is  cut  short.  Start  with  the  yes/no  questions  first  and  then 
move  into  the  open-ended  inquiries;  the  sequence  of  questions  should  reflect  a  logical  flow, 
a  smooth  transition  from  topic  to  topic.  Once  the  questionnaire  is  prepared,  conduct  a  dry 
run  with  your  staff  at  the  office.  Do  not  take  a  tape  recorder  to  the  interview;  your  subject 
will  talk  more  freely  if  he  does  not  feel  he  has  to  shape  each  sentence  just  so.  If  you  want 
to  retain  certain  specific  information  during  the  interview,  take  notes.  Later,  after  the  interview 
is  over  and  you  have  left  the  interview  location,  you  may  tape  your  impressions  so  they  are 
not  lost  to  time. 

f.  Develop  and  use  interviewing  skills.  (This  item  could  be  called  “interviewing  tricks  or  how 
to  loosen  a  sponsor  up.”)  Keep  these  points  in  mind.  If  the  prospective  sponsor  is  not  doing 
90  percent  of  the  talking,  you  are  losing  out.  Conduct  the  interview  in  the  sponsor’s  office 
(his  home  territory).  Show  the  sponsor  a  hard  copy  of  a  viewgraph  that  summarizes  his 
publications,  issues,  etc.  (this  is  one  way  to  indicate  to  him  that  you  have  done  your  homework 
and  that  you  are  familiar  with  his  situation).  Leave  him  something  like  a  data  plot,  photographs, 
or  charts.  Establish  him  as  the  expert.  Remember  lunches  are  good  for  establishing  rapport, 
but  are  not  the  place  for  hard  data  and  ideas.  Finally,  as  you  leave,  ask  him  about  other  people 
to  see  and  what  other  material  should  be  read. 
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Figure  4.1.  Windows  of  opportunity. 


The  following  are  suggested  strategies  for  winning: 

a.  Identify  new  “targets”  early.  You  cannot  get  enough  customer  contact,  and  when  you  do  make 
contact  communicate  in  the  customer’s  language  and  to  his  biases. 

b.  Select  the  targets  that  match  your  strengths. 

c.  Identify  your  weaknesses  and  start  correcting  them  by  training,  reading,  hiring,  and  teaming. 

d.  Know  your  customer’s  requirement/goals  better  than  he  does. 

e.  Determine  if  customer  considers  you  a  “front  runner”. 

f.  Obtain  management’s  commitment  and  devote  ample  resources  to  staff  a  good  proposal  team 
(red  team). 


4.4  THE  SYSTEM  ACQUISITION  PROCESS 

The  system  acquisition  process  is  presented  in  Figure  4.3.  The  process  is  discussed  in  terms  of 
DoD’s  research  and  development  categories:  research  (6.1),  exploratory  development  (6.2),  and  advanced 
development  (6.3).  The  last  mentioned  category  also  includes  advanced  technology  demonstrations  (6.3A). 
Helpful  definitions  are  included  as  Table  4.2,  while  Table  4.3  lists  useful  acronyms.  Figure  4.4  presents 
a  matrix  of  definitions  and  responsibilities  for  acquisition  categories  (ACAT). 

Figure  4.5  gives  an  overview  of  the  Navy  acquisition/decision  process  in  three  steps.  Figure  4.6 
and  Tables  4.4  and  4.5  show  the  people  involved  in  the  process:  overall  under  SECNAV,  the  resource 
sponsors,  and  claimants.  Finally,  Figure  4.7  illustrates  just  how  long  and  involved  the  process  can  be. 


4.5  DEVELOPING  PROPOSALS 

Figures  4.8  through  4.10  present  flowcharts  to  help  in  proposal  planning,  preparation,  and 
presentation,  respectively. 

The  method  of  proposal  development  recommended  in  this  section  comprises  the  development 
of  an  informal  proposal  followed  by  the  preparation  of  a  formal  proposal.  It  should  be  noted  that 
a  good  proposal  by  itself  is  insufficient;  a  bad  one  is  “lethal.” 

Be  aware  that  the  submission  of  an  informal  proposal  is  a  tacit  offer  to  commit  NOSC  resources; 
also  do  not  oversell  or  exaggerate  (it  is  counterproductive).  Keep  in  mind  that  the  informal  proposal 
creates  the  opportunity,  the  formal  proposal  establishes  the  contract.  For  the  informal  proposal  use 
B&P  funding  (department  or  major).  This  process  takes  time;  remember  that  your  goal  is  to  build 
a  client/advisor  versus  a  customer/salesman  relationship.  The  following  are  some  guidelines  for 
developing  a  successful  informal  proposal. 

a.  Make  sure  the  proposal  is  of  book  quality. 

b.  Develop  a  point  paper  or  self  narrated  brief.  It  must  skim  well.  Keep  it  short  and  profusely 
illustrated  (the  illustrations  should  pass  the  10-second  rule  and  have  instructive  captions).  It 
should  appeal  to  experts  as  well  as  laymen. 
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Figure  4.3.  System  acquisition  process. 


Table  4.2.  Definitions. 


Initiate:  Prepares  first  draft  of  document  and  generally  manages  review/comment  on  draft 

revisions. 

Submit:  Authorizes  insertion  of  smooth  document  into  final  review  and  approval  process. 

Chop:  An  indication  of  concurrence  by  initialing  in  an  indicated  place.  Not  used  in  smooth 

document  processing. 

Review:  An  indication  of  concurrence  by  signing  the  cover  sheet  in  the  indicated  place. 

Failure  to  sign  “as  reviewed"  stops  the  process. 

Approve:  Signature  making  the  document  an  official  paper  for  its  intended  use. 

Promulgate:  Process  of  ensuring  record  keeping  and  distribution  of  the  document  in  an  orderly 

and  centralized  manner. 


Table  4.3.  Acronyms. 


TOR 

Tentative  operational  requirement 

DOP 

Development  options  paper 

JMSNS 

Justification  of  major  system  new  start 

OR 

Operational  requirement 

DCP 

Decision  coordination  paper 

SCP 

System  concept  paper 

NDCP 

Navy  decision  coordinating  paper 

TEMP 

Test  and  evaluation  master  plan 

Acquisition 

Categories  Documentation 

(ACAT)  Criteria  Decision  Requirements 

I  Major  Programs:  SCP 

•  200M  RDT&E  SECDEF  DCP 

IB  prod  DEPSECDEF  and 

•  National  urgency  TEMP 

•  As  directed 


II  S  Less-Than  Major:  SECNAV  NDCP 

- •  100M  RDT&E  -  & 

IIC  500M  prod  CNO  TEMP 


IIC  500M  prod  CNO  TEMP 


Ill 

•  Significant  impact  on 

OPNAV 

TEMP 

military  characteristics 

Sponsor 

IV  T 

•  All  other  acquisitions 

•  OPEVAL  required 

SYSCOM  CMDR 

TEMP 

IV  M 

All  others 

SYSCOM  CMDR 

TEMP 

Figure  4.4.  Definitions  and  responsibilities  for  acquisition  categories  (ACAT). 
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Figure  4.5.  Process  overview  (I  of  3). 


Figure  4.5.  Process  overview 


Table  4.4.  Resource  sponsors  and  tasks 


Undersea  surveillance/oceanography 


Platform  Sponsors 

.  OP-02 

Aviation . 

. OP-05 

.OP-03 

Support  Sponsors 

.OP-01 

R&D . 

. OP-098 

.  OP-04 

C5  . 

. OP-094 

OP-095 

Command/administration . 

. OP -098 

OP-093 

Military  assistance . 

. OP-08 

OP -009 

Plans,  policy,  operations  . 

. OP-06 

Tasks 

Develop  programs 
Participate  in  appraisals/CPAMs 
Update  data  base 


Table  4.5.  Claimants 

Have  primary  responsibility  for  program  execution 

Submit  suggested  programs  to  sponsors 

NAVMAT/SYSCOMs  provide  pricing  estimates 

Review  and  comment  on  proposed  program 
CINCs  strategy  review1 

RAD  II  (Oct  FYDP)  and  RAD  IV  (Jan  FYDP)  scrub 

Break  down  (to  UIC  level)  appropriation  totals  for  Oct/Jan  internal  Navy  five-year  plans 
Optional  claimant  input  to  identify  issues 

SPPs  written  by  resource  sponsors  to  define  program  changes  in  response  to  claimant  input 


Figure  4.9.  Proposal  preparation 


Figure  4.10.  Proposal  presemation. 


c.  Remember  that  the  proposal  teaches  what  it's  all  about.  It  establishes  or  creates  the  need, 
and  then  proposes  the  solution  and  points  out  the  opportunity.  It  establishes  your  theme  by 
addressing  the  superiority  of  approach;  “doability”;  credibility  (why  us?);  risk,  timing  and 
costs,  and  management  commitment;  the  deliverable  (your  defined  product);  and  the  early 
program  involvement. 

d.  Develop  your  briefing.  This  is  very  important  —  the  superiority  of  an  idea  or  design  is  not 
so  self-evident  that  it  does  not  require  supporting  data  and  presentations.  So  prepare  a  first- 
class  briefing  designed  for  dialogue  with  the  customer  (you  want  to  incorporate  and  address 
his  concerns).  Your  speaker  should  be  “quick  on  his  feet”  and  be  accompanied  by  those  who 
have  the  expertise  to  field  difficult  questions.  In  this  presentation  you  want  to  avoid  sales 
soliloquies. 

e.  Bring,  if  possible,  any  of  the  following  items  (they  are  listed  in  order  of  priority): 

Hardware 

Mockups 

Photos 

Engineering  type  drawings 

Artist  sketches 

Reports. 

f.  Sell  your  theme  emphasizing  “we  are  further  along.” 

g.  Detect/create  a  need/solution  your  customer  did  not  know  he  had  and  is  not  getting  from 
the  competition. 

h.  Offer  something  he  cannot  get  from  the  competition  (experience,  Fleet/intel  access,  DARPA 
contracting  authority,  staff  support,  etc.).  Items  g  and  h  both  emphasize  the  fact  you  should 
know  your  competition.  Know  who  they  axe,  their  strengths,  weaknesses,  tactics,  and  pricing 
information. 

i.  Know  the  leverage  points  (i.e.  critical  thresholds,  costs,  etc.). 

j.  Cultivate  a  champion  at  headquarters. 

k.  Dry-run/murder-board  proposal. 

l.  Offer  to  assist  the  sponsor  (ghostwrite  a  TDR,  etc.). 

Past  experience  has  demonstrated  several  things  to  keep  in  mind.  Prior  experience  is  less  important 
than  the  best  proposal.  At  this  stage  time  spent  on  a  good  presentation  pays  off  more  than  additional 
analysis.  Our  biggest  mistake  is  to  try  and  do  it  all  (all  of  us  are  not  Thomas  Edisons). 

That  brings  us  to  formal  proposal  preparation.  The  following  observations  are  intended  to  provide 
some  help  in  approaching  the  formal  proposal. 

a.  Currently  NOSC  does  not  have  an  instruction/manual. 

b.  NELCINST  5000.2A  is  out  of  date  (but  not  bad). 

c.  Most  NOSC  proposals  will  use  DD  form  1498  (LPS). 

See  department  administration  officer. 

Seek  Code  121  help  (COMSTOCK). 

d.  It  is  important  to  involve  affected  codes. 

Verify  manpower  availability. 

Verify  costs/schedule. 

Obtain  necessary  signatures. 


e.  Cover  letter  most  helpful  (signed  out  as  high  as  possible). 

f.  Large  programs  will  be  done  via  by  letters  of  agreement:  assignments  of  responsibility  (e.g., 
deputy  PM,  lead  lab,  etc.). 

Figure  4.11  presents  the  types  of  statements  of  work  (SOWs)  and  their  associated  contract  documents. 

4.6  NEW  MARKETING  STRATEGIES 

The  new  marketing  strategies  for  NOSC  include  consideration  of  the  following  items: 

a.  Submit  requirement  via  CINCs  (support  your  local  NSAP  office). 

b.  Showcase  system  in  war  games. 

c.  Participate  in  high  level  studies,  appraisals,  strategies,  master  plans. 

d.  Support  your  sponsor  ( NSTEP  =  agent  in  place). 

e.  Go  black  . . .  (jumps  over  many  wickets). 

f.  End-runs  -  caution :  very  high  risk  for  Center/CO  &  TD  (safer:  DSB/NRAC/etc.). 

g.  Bailing  out  troubled  programs. 

h.  Advanced  tech  demos  (OP-98). 

i.  Avoid  joint  programs. 

j.  Top-level  warfare  requirements  (TLWR)/battle  force  integration  management  (BFIM)  & 
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SECTION  5 

STAFFING,  TEAM  BUILDING,  AND  COMMUNICATION 
F.  Gordon,  Code  60 

5.1  INTRODUCTION 

5.1.1  References 

In  Search  of  Excellence 
The  One-Minute  Manager 
Negotiating  to  Yes 

How  to  Develop  Your  Executive  Ability 

5.1.2  Outline 

Introduction 

References 

Outline 

Summary 

Staffing,  Team  Building,  and  Communication 
Staffing 
Team  Building 
Communication 
Implementation/Execution 

5.1.3  Summary 
See  below. 


5.2  STAFFING,  TEAM  BUILDING,  AND  COMMUNICATION 

What  do  we  mean  by  staffing,  team  building,  and  communication?  Briefly  stated  our  subjects 
can  be  brought  together  as  follows:  enough  people,  with  the  right  expertise  when  you  need  them,  working 
harmoniously  toward  a  common  set  of  goals  with  greatest  efficiency. 

In  the  following  pages  each  of  the  elements  of  our  topic  will  be  discussed.  In  staffing  (“enough 
people,  with  the  right  expertise  when  you  need  them .  .  .  ”)  you  are  looking  for  people  with  the  following 
characteristics: 

Motivated 

Educated 

Available 

Affordable 

Experienced 

High  priority  for  your  requirements. 


In  team  building  (your  staff  “working  harmoniously  toward  a  common  set  of  goals. . .  ’’)  it  is 
important  that  you  keep  the  following  ideals  in  mind: 

Goals  of  the  project  placed  above  self-interest 

Mutual  trust  and  respect 

Everyone  contributes. 

And,  finally  in  communication  (your  staff  “working  harmoniously  . . .  with  greatest  efficiency”) 
open  communication  is  a  necessity;  this  open  communication  includes  listening  and  understanding, 
has  no  filters,  and  holds  no  untoward  surprises. 

You  program  managers  have  reason  for  optimism:  NOSC  hires  good  people  who  are  educated, 
motivated,  and  experienced.  Your  job  is  to  shape  them  into  a  team,  motivate  them,  and  lead  them. 
The  bottom  line  is  that  good  managers  have  good  people  working  for  them. 

What  is  the  secret  then  of  being  a  good  manager?  First,  it  is  hard  work  on  your  part.  Second, 
it  is  the  consistent  application  of  general  management  methods  that  work.  Third,  it  is  hard  work. 

The  following  subsections,  though  still  brief,  discuss  staffing,  team  building,  and  communication 
in  more  detail. 

5.2.1  Staffing 

Staffing,  in  most  instances,  is  the  easiest  part.  We  can  do  it  by  the  numbers,  acquiring  our  personnel 
from  coworkers  at  NOSC,  other  Navy  laboratories,  and  support  contractors.  There  is  a  real  need  to 
identify  your  technical  expert  so  you  have  an  authoritative  viewpoint  early  in  your  program.  As  you 
build  your  staff  it  is  helpful  to  keep  in  mind  that  for  many  jobs  inner  drive  and  motivation  are  more 
important  than  genius. 

It  is  also  important  that  you  do  not  forget  support  staff;  this  includes  the  Supply  and  Accounting 
staffs  here  at  NOSC,  already  in  place  to  help  implement  your  programs.  You  will  probably  find  that 
your  biggest  problems  are  not  technical.  Most  programs  experience  problems  related  to  contracting, 
and  “system”  constraints  probably  exceed  technological  problems.  This  might  be  related  to  the  fact 
that  in  the  Library  of  Congress  there  are  1,152  lineal  feet  of  documents  governing  the  supply/acquisition 
process.  The  NOSC  Supply  and  Accounting  staffs  will  be  your  trailblazers  through  this  acquisition  jungle. 

5.2.2  Team  Building 

5.2.2. 1  The  Basic  Rule  Put  simply,  the  rule  says  do  not  mess  with  human  nature.  Human  nature 
reflects  the  law  of  egocentrism:  each  person  is,  and  regards  himself  as,  the  center  of  his  own  world 
of  experience  and  action.  This  can  be  seen  in  the  way  people  see  themselves.  A  self-assessment  performed 
by  a  random  sample  of  100  males  produced  the  following  results: 

a.  Ability  to  get  along  with  others 

All  100  ranked  themselves  in  the  top  50  percent  of  the  population 
60  percent  ranked  themselves  in  the  top  10  percent  of  the  population 
25  percent  ranked  themselves  in  top  1  percent  of  the  population 


b.  Leadership 

70  percent  rated  themselves  in  the  top  25  percent  of  the  population 
2  percent  felt  they  were  below  average  as  leaders 

c.  Athletic  ability 

60  percent  ranked  themselves  in  the  top  25  percent  of  the  population 
6  percent  indicated  they  were  below  average. 

5. 2.2. 2  Motivation  It  is  best  to  recognize  what  human  nature  is  and  proceed  from  there.  Getting 
along  with  human  nature  requires  that  we  recognize  that  people  do  things  because  its  in  their  best  interest 
to  do  them.  Thus,  be  aware  of  the  following  motivating  factors: 

Self-fulfillment 

Anticipated  satisfaction  in  achievement 
Recognition  and  respect 
Opportunity  to  contribute 
Accountability  and  trust-expectations. 

Interestingly,  pay  is  not  at  the  top  of  the  list. 

Figures  5.1  through  5.5  present  the  Just  in  Time  (JIT)  MK46  production  line  case  history  that 
demonstrates  how  motivating  factors  can  be  applied  effectively. 

5.2.2.3  Decisionmaking  The  basic  practice  to  remember  here  is  do  not  make  all  the  decisions  yourself. 
If  you  feel  you  must  make  every  decision,  you  merely  limit  the  quality  of  your  program,  ensure  that 
nothing  happens  when  you  are  gone,  and  fail  to  build  the  sense  of  ownership  in  the  rest  of  your  team. 
As  you  allocate  decisionmaking  responsibilities  you  will  see  that  confidence  inspires  confidence  and 
success  breeds  success.  The  program  will  be  the  winner. 

5. 2. 2. 4  Modifying  Behavior  Whenever  there  is  a  team  effort  there  is  bound  to  come  a  time  when, 
for  whatever  reasons,  some  team  member  is  performing  his  or  her  task  at  a  minimal  or  subminimal 
level.  You  as  program  manager  will  need  to  and,  it  is  hoped,  want  to  address  this  problem.  The  first 
step,  often  forgotten,  is  to  praise  good  work  and  behavior.  Secondly,  in  contrast  to  the  counsel  of 
The  One-Minute  Manager,  use  criticism  sparingly.  If  you  must  criticize,  never  criticize  a  team  member 
in  front  of  others  nor  in  an  emotional  outburst.  Also  focus  on  criticizing  the  act,  not  the  individual. 
Explore  ways  of  saying  what  needs  to  be  said;  for  instance,  the  use  of  the  phrase  “are  you  aware. .  . 
can  provide  a  reasonably  comfortable  transition  into  discussing  a  possible  area  of  trouble. 

5. 2. 2. 5  Conflict  Management  There  is  another  inevitability  when  two  or  more  human  beings  are 
working  together  for  any  length  of  time.  There  will  be  some  conflict.  The  law  of  egocentrism  is  still 
at  work.  The  following  are  some  useful  preliminary  steps  to  managing  conflicts: 

Get  the  facts 

Separate  the  emotion  from  the  problem 
Listen  to  understand 
Look  for  points  of  agreement 
Look  for  graceful  ways  out. 

It  would  be  very  useful  to  familiarize  yourself  with  the  negotiating  approaches  presented  in  Table 
5.1,  and  practice  them  as  you  have  opportunitv. 


FINAL  ASSEMBLY  -  FIRE  CONTROL 
(BEFORE  JIT) 


Figure  5.2.  Process  flow  diagram. 


Figure  S.3.  JIT  process  flow  diagram. 


VISUAL  DEFECTS 


Figure  5.4.  MK  46  quality  trends. 


5.5.  Benefits  of  JIT. 


Problem 

Positional  bargaining:  Which  game  should  you  play? 


SOFT 

Participants  are  friends 
The  goal  is  agreement 


Make  concessions  to 
cultivate  the  relationship 

Be  soft  on  the  people  and 
the  problem 

Trust  others 

Change  your  position  early 

Make  offers 

Disclose  your  bottom  line 

Accept  one-sided  losses  to 
reach  agreement 

Search  for  the  single  answer: 
the  one  they  will  accept 

Insist  on  agreement 

Try  to  avoid  a  contest  of 
will 

Yield  to  pressure 


Table  S.l.  Negotiating  approaches 


HARD 

Participants  are  adversaries 

The  goal  is  victory 

Demand  concessions  as  a 
condition  of  the 
relationship 

Be  hard  on  the  problem  and 
the  people 

Distrust  others 

Dig  in  to  your  position 

Make  threats 

Mislead  as  to  your  bottom 
line 

Demand  one-sided  gains  as 
the  price  of  agreement 

Search  for  the  single  answer: 
the  one  you  will  accept 

Insist  on  your  position 

Try  to  win  a  contest  of  will 

Apply  pressure 


Solution 

Change  the  Game  — 
Negotiate  on  the  Merits 


PRINCIPLED 

Participants  are  problem- 
solvers 

The  goal  is  a  wise  outcome 
reached  efficiently  and 
amicably 

Separate  the  people  from  the 
problem 

Be  soft  on  the  people,  hard 
on  the  problem 

Proceed  independent  of  trust 

Focus  on  interests,  not 
positions 

Explore  interests 

Avoid  having  a  bottom  line 

Invent  options  for  mutual 
gain 

Develop  multiple  options  to 
choose  from;  decide  later 

Insist  on  using  objective 
criteria 

Try  to  reach  a  result  based 
on  standards  independent 
of  will 

Reason  and  be  open  to 
reason;  yield  to  principle, 
not  pressure 


5. 2. 2. 6  The  Basic  Rule  and  You.  Remember  that  the  law  of  egocentrism  applies  to  you  too.  Thus, 
it  is  in  your  best  interest  to  develop  a  strong  team.  Finally,  recognize  that  the  same  factors  that  motivate 
you  motivate  your  team  as  well.  The  particulars  may  be  different,  but  the  principles  remain  the  same. 

5.2.3  Communication 

Whenever  you  have  a  group  of  people  working  together  there  will  be  communication.  The  question 
is,  however,  will  it  be  good  communication  or  poor  communication?  This  is  the  choice.  Will  we  have 
open,  straightforward,  two-way  communication?  Or  will  we  have  limited,  one-way  communication 
fueled  by  rumor? 

We  know  that  good  communication  is  required  for  effective  team  building  and  project  efficiency 
and  that  the  lack  of  good  communication  leads  to  poor  efficiency.  People  worry  if  they  think  there 
is  something  that  will  affect  them  that  they  do  not  know.  They  worry  about  their  own  self-interests, 
and  worry  is  more  likely  to  be  caused  by  rumor  than  fact.  There  can  even  be  a  geographical  or  location 
component  to  the  communication;  Figure  5.6  was  taken  from  In  Search  of  Excellence.  Good 
communication  takes  work. 

Good  program  communication  can  be  promoted  through  implementing  the  following  approaches: 

Regular  meetings 

Management  by  walking  around  (not  announced,  scheduled  tours) 

Tell  them  more  than  they  need  to  know,  let  them  filter  for  themselves 

Listen,  consider,  evaluate,  discuss 

Remember  to  cooperate  with  human  nature 

Do  not  shoot  the  messenger  (value  the  person  who  tells  you  hard  truths). 


5.3  IMPLEMENTATION/EXECUTION 

The  bottom  line  is  that  management  techniques  are  relatively  simple  to  delineate  and  learn.  The 
key  then  to  good  program  management  is  implentation  and  execution. 

The  authors  of  In  Search  of  Excellence  (Chapter  1),  in  reviewing  studies  of  62  successful  companies, 
found  many  similarities  among  them.  These  companies  exhibited  the  following  characteristics  from 
1961  through  1980: 

Compound  asset  growth 

Compound  equity  growth 

Highest  average  ratio  of  market  value  to  book  value 

Highest  average  return  on  capital 

Highest  average  return  on  equity 

Highest  return  on  sales. 

Table  5.2  presents  the  eight  common  basic  principles  identified  as  the  attributes  that  “characterize 
most  nearly  the  distinction  of  the  excellent,  innovative  companies  ’’ 

A  final  word.  The  principles  governing  staffing,  team  building,  and  communication  are  well  known. 
It  is  your  task  to  work  at  implementing  them  for  your  program  and  learning  how  they  work  in  the 
real  world.  Then  you  can  also  model  them  and  teach  them  to  vour  coworkers. 


»V  V  f  / 


Source:  In  Search  of  Excellence 


Table  5.2.  Principles  of  excellence  and  innovation. 


Principle 


Comments 


1.  A  Bias  for  Action 


2.  Stay  Close  to  the  Customer 


3.  Autonomy  and 
Entrepreneurship 


4.  Productivity  through  People 


I 

5.  Hands-on,  Value  Driven 


I 

6.  Stick  to  the  Knitting 

7.  Simple  Form,  Lean 
Administrative  Staff 


I 


8.  Simultaneous  Loose-Tight 
Properties 


Break  the  problem  into  parts  (chunking) 

Use  small  Ad  Hoc  task  forces  with  limited  life,  (specific 
assignment  for  a  short  period  of  time) 

“Do  it,  fix  it,  try  it”  —  characterizes  experimenting 
organizations 
Simplify 

Chaotic  actions  are  better  than  inaction 
Ready,  fire,  aim,  learn 

Figure  out  what  he  needs 

Provide  it 

Quality 

Nichemanship 

Listen  to  the  user 

Break  the  corporation  into  small  companies 
Encourage  them  to  think  independently  and  competitively 
Support  innovation 
Communicate 

“The  new  idea  either  finds  a  champion  or  dies...” 
Edward  Schon,  MIT 

Create  an  awareness  that  their  best  efforts  are  essential 
Success  will  be  recognized 
Focus  on  people  —  build  a  team 

Managers  keep  in  touch 
A  belief  in  being  the  “best” 

A  belief  in  the  details  of  execution 
A  belief  in  the  importance  of  people 
A  belief  in  superior  quality  and  service 
Practice  management  by  walking  around 

Build  diversification  strategies  on  some  central  skill  or  strength 

Minimum  staff  at  the  corporate  level 
Subunits  should  have  their  own  staff  support  (supply, 
personnel,  finance) 

Simple  organizations  —  accountability  and  autonomy 

Coexistence  of  firm  central  direction  and  maximum  individual 
autonomy,  entrepreneurship,  and  innovation 
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SECTION  6 

COMPUTERIZED  ASSISTANCE 
T.  Keil,  Code  9101 


6.1  INTRODUCTION 


6.1.1  References 

None. 

6.1.2  Outline 

Introduction 

References 

Outline 

Summary 

What  Computer  Assistance  is  Available 
PCs  and  Minicomputers 
Effective  Use  of  Electronic  Mail 
User  Assistance — What  Help  is  There? 

Future  Developments 

6.1.3  Summary 

There  is  a  great  explosion  in  the  number  of  inexpensive  desktop  computers  being  used  at  NOSC. 
By  the  end  of  this  year  there  will  be  approximately  one  personal  computer  (PC)  per  employee.  It  is 
very  difficult  even  for  those  in  the  field  to  maintain  an  awareness  of  the  advances  in  hardware  and 
software.  We  are  all  faced  with  the  associated  problems:  these  include  what  to  buy,  how  to  use  it, 
how  to  use  it  on  the  GCB  (our  local  NOSC  network),  and  so  forth. 

Therefore,  this  presentation  has  a  dual  purpose.  First,  it  will  make  you  aware  that  computerized 
tools  are  available  to  the  project  manager.  Second,  it  will  identify  the  sources  that  will  help  you  determine 
exactly  what  tools  are  available  and  that  will  help  you  use  these  tools  effectively. 


6.2  WHAT  COMPUTERIZED  ASSISTANCE  IS  AVAILABLE? 

Computer  resources  at  NOSC  fall  into  two  categories: 

a.  Centralized  (the  computer  center,  administrative  and  STAFS,  and  the  centerwide  office 
automation  network  (COAN) 

b.  Division  and  project  computers  (minicomputers  running  UNIX  and  VMS  and  microcomputer'' 


With  the  explosion  in  microcomputers  the  traditional  role  of  a  centralized  computer  center  is  changing 
from  being  a  computing  resource  to  being  a  database  and  routing  resource  with  the  micros  becoming 
the  workhorses. 

What  I  will  discuss  is  how  the  micro  (PC)  can  be  used  effectively  with  the  database  and  routing 
resources  (minicomputers)  we  have  here  at  NOSC: 

a.  Accessing  minis  from  a  PC 

b.  How  to  use  electronic  mail 

c.  Where  to  go  for  help  and  instruction 

d.  What  changes  are  coming  in  the  future  to  make  work  easier. 

6.2.1  PCs  and  Minicomputers 

NOSC  has  in  most  buildings  a  generalized  communication  backbone  (GCB)  which  allows 
communication  between  computers.  With  this  network  it  is  possible  to  connect  directly  to  a  minicomputer 
and  work  on  the  minicomputer  or  to  transfer  files  (data  and  information)  between  the  microcomputer 
and  the  minicomputer.  We  will  discuss  PC-mini  communication  and  the  GCB. 

6.2.2  Effective  Use  of  Electronic  Mail 

Electronic  mail  has  become  one  of  the  most  popular  uses  of  computer  networks  (connecting 
computers).  We  will  discuss  the  following: 

a.  How  electronic  mail  can  be  used  effectively 

b.  What  networks  are  available  to  users  (to  send  information  to  sponsors  for  instance) 

c.  How  you  access  NOSC  computers  while  on  travel. 

6.2.3  The  Project  Management  Support  System  (PMSS) 

The  PMSS,  administered  by  NOSC  Code  1 1 ,  is  designed  to  provide  NOSC  program  managers 
and  support  personnel  with  an  easy-to-use  approach  for  accessing  project  information.  Among  its 
offerings  are  software,  hardware,  documentation,  training,  and  one-to-one  consultation.  PMSS  currently 
provides  the  following  capabilities: 

Access  to  an  online  database  containing  financial  data 
Access  to  commercial  software  tools 
Electronic  mail. 

PMSS  was  developed  to  support  both  the  novice  and  the  more  experienced  computer  user.  PMSS 
guides  each  user  with  a  series  of  choices,  or  menus.  When  you  enter  PMSS,  you  see  a  list  of  options 
on  the  menu.  Based  on  your  selection,  you  may  see  your  present  menu  replaced  by  another  menu, 
or  you  may  be  asked  for  specific  information,  such  as  a  job  order  number.  You  can  select  online  help 
or  exit  PMSS  from  every  menu. 

6.2.4  User  Assistance — What  Help  is  There? 

User  assistance  is  available  for  both  hardware  and  software  problems:  “test  driv  ing”  the  hardware 
and  software,  help  in  purchasing  it,  installing  it,  repairing  it,  getting  training  on  how  to  use  it,  and 


obtaining  assistance  in  doing  small  programming  tasks  so  the  software  can  be  used  more  effectively. 
The  assistance  available  and  contacts  will  be  discussed. 

6.2.5  Future  Developments 

There  are  several  projects  underway  that  will  enable  personnel  to  use  PCs  and  minis  together  more 
effectively.  We  will  discuss  several  of  these: 

a.  Development  of  a  simple  micro  mail  and  file  transfer  capability  for  the  micro-only  user 

b.  An  integrated  NOSC  mail  system  where  everyone  has  a  mail  name 

c.  Development  of  electronic  paperwork  tools  to  allow  many  paperwork  functions  to  be  done 
electronically — stubs  for  instance 

d.  Development  of  a  reliable  electronic  signature  system  which  will  guarantee  that  the  person  whose 
name  appears  did  in  fact  “sign”  the  document. 
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SECTION  7 

PLANNING,  SCHEDULING,  AND  ASSESSMENT 
R.  Nees,  Code  805 

7.1  INTRODUCTION 

7.1.1  References 

There  is  a  multitude  of  formal  planning  documents  that  make  up  the  Navy  RDT&E  planning  process. 
Defense  Guidance  (DG)  documents 
Technology  and  Description  (TAD)  document 
Joint  Long  Range  Strategic  Appraisal  (JLRSA)  document 
Joint  Strategic  Planning  Document  (JSPD) 

Joint  Program  Assessment  Memorandum  (JPAM) 

DON  Policy  and  Planning  Guidance  (DNPPG) 

CNO  Policy  and  Planning  Guidance  (CPPG) 

CNO  Program  Analysis  Memorandum  (CPAM) 

Program  Objective  Memorandum  (POM) 

Department  of  the  Navy  Five-Year  Program  (DNFYP) 

R&D  Plan 

Science  and  Technology  Objective  (STO) 

Operational  Requirement  (OR) 

Marine  Corps  related  documents 
Department  of  the  Navy  RDT&E  Management  Guide 
NAVSO  P  2457 

Navy  System  Management  instructions  used  by  NOSC 
Major  Systems  Acquisition  (DODIR  5000.1) 

RDT&E  Acquisition  Procedures  (OPNAV1NST  5000. 42B) 

Project  Master  Plan  (NAVMATINST  5200.1  IB) 

Integrated  Logistics  Support  (NAVMATINST  4000. 20B) 

Configuration  Management  (NAVMATINST  4130. 1A  and  OPNAVINSTs  4130. 1  and  4130.2) 
Test  and  Evaluation  (OPNAVINST  3960. 10B) 

Design  Requirements  Baseline  (NAVMATINST  41 30.1  A) 

7.1.2  Outline 

Introduction 

References 

Outline 

Summary 

General 

Planning  as  a  Tool 
Project  Management  Plan 
Integrated  Planning 
Milestones 

Technical  Sequence  and  Flow— Networks 
Work  Breakdown  Structure 


Automated  Work  Breakdown  Structure  System 
WBS  Element  Description  Record 

Project  Planning  Range 

Schedules 
Bar  Charts 
Gantt  Charts 
Line  Chans 

Project  Progress  Reports 
General 
Objectives 
Procedure 

Procurement  Planning 
Contract/ Purchase  Order  Planning 
Acquisition  Plan  Requirement 

7.1.3  Summary 

See  below. 


7.2  GENERAL 

Project  management  is  performed  at  NAVOCEANSYSCEN  by  an  individual  with  the  title  of 
“Project  Manager.”  The  project  manager  has  specific  responsibility  for  achieving  project  objectives 
and  accomplishing  project  assignments.  Any  manager  who  is  the  cognizant  individual  responsible  for 
performance  on  a  project  is  termed  “project  manager.”  The  extent  that  a  project  manager  becomes 
involved  in  managing  a  project  depends  on  its  size.  Some  of  the  smaller  projects  in  the  Center  may 
not  require  the  many  formal  approaches  for  project  management  that  a  larger  project  may  require. 
The  project  manager  should  tailor  his  requirements  to  the  needs  of  the  specific  project.  Large  projects, 
however,  should  use  the  guidelines  as  delineated  in  this  section. 

The  project  manager  and  personnel  assigned  under  his  cognizance  provide  for  the  development 
of  project  technical  and  administrative  plans,  procedures,  and  practices  as  necessary  to  facilitate  a  high 
degree  of  technical  and  management  performance.  These  plans  and  procedures  should  mesh  harmoniously 
with  other  Center  practices  and  procedures.  Procedures  should  be  established  and  maintained  for 
obtaining  operational  and  functional  support  from  a  variety  of  resources  such  as  other  departments, 
other  government  activities,  and  independent  contractors.  Project  managers  have  the  responsibility 
for  planning  and  administering  assigned  resources  within  approved  project  and  operational  budgets. 
They  approve  estimates  of  funding  requirements  prior  to  incorporation  into  project  budgets.  Proper 
planning  will  set  the  stage  for  allocation  of  resources  to  the  project  and  ensure  that  plans,  programs, 
budgets,  and  schedules  are  properly  integrated  and  time  phased.  In  addition,  initial  project  planning 
should  consider  the  establishment  of  a  complete  project  chronological  history  that  ensures  the  availability 
of  accurate  information  concerning  all  significant  events  and  decisions  relating  to  the  project,  and  from 
which  the  project  can  be  reconstructed  step  by  step.  The  project  manager  should  be  aware  of  the  current 
status  and  progress  of  the  assigned  project  and  be  able  to  convey  that  information  to  appropriate  Center 
and  higher  command  officials  as  may  be  required.  Technical  content  of  reports  and  quality  of  technical 
data  should  be  reviewed  to  assure  that  high  professional  standards  are  maintained.  Security  precautions 
in  accordance  with  DoD  policy  should  be  strictly  followed  and  enforced. 
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Project  managers  assume  responsibility  for  the  management,  planning,  direction,  and  control  of 
project  resources  necessary  for  project  completion.  Project  managers  have  specific  authority  and 
responsibility  for  directing  a  system’s  overall  progress  through  the  conceptual,  validation,  full-scale 
development,  production,  deployment,  and  Fleet  support  phases  of  Navy  weapons  system  acquisition 
and/or  the  completion  of  the  project. 

The  following  documents  should  be  prepared,  as  necessary,  for  each  project: 

a.  Project  functions 

b.  Project  organization  chart 

c.  Organizational  relationship  chart 

d.  Functional  areas  of  responsibility 

e.  Financial  plan 

f.  Milestone  charts 

g.  Network  plans  and/or  schedules 

h.  Work  breakdown  structure 

i.  Acquisition  plan 

j.  Value  engineering  plan 

k.  Integrated  logistics  support  plan 

l.  Configuration  management  plan 

m.  Test  and  evaluation  plan 

n.  Security  plan 

o.  Training  plan 

p.  Safety  plan 

q.  Quality  assurance  plan 

r.  Data  management  plan 

s.  Human  engineering  plan 

t.  Project  master  plan 

u.  Project  management  plan. 

A  primary  function  of  project  management  is  the  implementation  of  a  useful  planning  and  control 
system.  Control  systems  are  required  to  ensure  that  allocation  of  resources  and/or  acquisition  of  services 
and  equipment  stay  within  the  limits  of  planned  resources.  The  management  planning  and  control 
concepts  discussed  herein  have  been  in  existence  for  many  years  in  both  government  and  industrial 
organizations.  These  techniques  provide  for  timely  and  accurate  management  decisionmaking. 

7.3  PLANNING  AS  A  TOOL 

Project  plans  are  necessary  to  facilitate  an  organized,  mutually  supportive  set  of  documents  which 
translate  program  authorization,  control,  and  visibility  into  easily  understood  road  maps.  The 


documentation  should  be  designed  so  as  to  enhance  the  ability  of  management  to  react  in  a  timely 

and  favorable  manner  with  regard  to  the  direction  of  the  project.  £ 

The  techniques  used  in  planning  should  consider  elements  such  as  scope  and  complexity  of  the 
project,  available  project  information  and  data,  and  project  commitments.  Also,  the  cost  of  providing 
control  and  reporting  documentation  must  be  considered.  Any  such  documentation  must  be  clearly 
specified  in  advance  so  that  its  cost  can  be  included  in  the  overall  project  funding  requirements. 

Choosing  proper  management  techniques  early  in  project  development  enhances  the  capability 
of  management  to  attain  project  objectives  and  goals.  One  technique  that  is  commonly  used  is  termed 
“management  by  exception.”  Management  by  exception  is  defined  as  the  comparing  of  project  plans 
and  status  with  desired  results  for  the  purpose  of  correcting  those  conditions  that  do  not  support  overall 
objectives  and  goals.  By  employing  management  by  exception  principles,  management  is  able  to  spend 
a  greater  amount  of  time  in  areas  that  offer  the  probability  of  attaining  maximum  gain  to  the  project. 

Another  technique  commonly  used  is  termed  “management  by  objectives.”  Management  by  objectives 
may  be  defined  as  the  direction  of  resources  towards  the  accomplishment  of  planned  goals.  The  functions 
of  management  by  objectives  are  planning,  organizing,  staffing,  directing,  and  controlling.  The 
attainment  of  planned  goals  is  contingent  upon  clear,  concise,  and  well  understood  policies  designed 
to  enhance  goals  which  are  important  and  different.  Rewards  will  flow  from  accomplishing  the 
extraordinary  goal. 


7.4  PROJECT  MANAGEMENT  PLAN 

The  project  management  plan  should  specify  an  introduction,  approach,  project  management 
organizational  structure  and  management  controls,  milestones  and  schedules,  financial  plan,  and  work 
breakdown  structure.  An  example  of  a  project  management  plan  is  offered  in  Appendix  7A.  W& 


7.5  INTEGRATED  PLANNING 

Integrated  planning  sessions  are  one  of  the  most  important  tools  available  to  project  management. 
All  project  management  personnel  should  be  trained  in  management  techniques  that  enhance  integrated 
planning.  Periodic  planning  sessions  that  consist  of  leading  personnel  through  all  disciplines  related 
to  the  project  are  necessary  during  project  initiation,  development,  test  and  evaluation,  and  acceptance. 
Planning  sessions  that  generate  networking  of  activities  and  events,  establish  technical  sequence  and 
flow  constraints,  and  identify  critical  paths  provide  the  basis  for  allocating  resources  and  planning 
realistic  time  frames  for  meeting  project  objectives.  Participants  in  integrated  planning  sessions  should 
be  alert  to  external  constraints  that  may  impact  on  the  timely  completion  of  the  project.  Networking, 
through  planning  sessions,  establishes  the  critical  path  of  the  project  and  permits  planning  for  resources 
to  improve  or  alleviate  any  identified  slippage  in  the  project  schedule. 

Task  responsibility  matrices  are  necessary  to  assure  that  all  segments  of  the  project  development 
effort  have  been  effectively  assigned.  The  matrix  should  identify  with  both  technical  personnel 
assignments  and  the  technical  effort  involved  in  the  assignment.  Supportive  documents  such  as  the 
project  organizational  structure  and  the  work  breakdown  structure  should  be  used  in  developing  an 
effective  responsibility  matrix. 
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7.6  MILESTONES 


Establishing  milestones  for  each  major  segment  of  a  system  under  development  is  termed  milestone 
scheduling.  Milestones  are  similar  to  network  events  in  that  both  reflect  points  in  time.  The  term 
“milestones”  is  used  to  identify  significant  events  that  must  be  accomplished  in  order  to  complete  the 
project  successfully.  Milestone  dates  may  be  established  for  both  start  and  completion  events  of  activities. 
Accomplishment  of  a  significant  event  signifies  a  completed  milestone.  Milestones  may  also  be  portrayed 
on  Gantt  charts,  networks,  and  line  charts. 

Milestone  dates  are  established  from  significant  events  that  must  occur  to  complete  the  project 
successfully.  The  number  of  milestones  chosen  for  scheduling  and  control  purposes  depends  on  the 
size  of  the  project.  Ordinarily,  at  least  2  weeks  should  separate  milestone  events.  Milestones  are  to 
reflect  those  events  that,  if  not  accomplished  on  schedule,  will  have  significant  impact  on  the  project. 
One  of  the  most  important  functions  of  project  management  is  to  assure  that  milestones  dates  are 
negotiated  and  realistically  established  in  consonance  with  both  internal  and  external  project  requirements. 

The  following  typical  milestone  events  are  provided  as  examples: 

a.  Obtain  project  go-ahead 

b.  Start  functional  specifications 

c.  Functional  specifications  complete 

d.  Start  system  design 

e.  System  design  complete 

f.  Start  procurement 

g.  Start  system  software  development 

h.  Start  fabrication 

i.  Contractor  delivers  hardware 

j.  Fabrication  complete 

k.  Start  assembly  of  system 

l.  System  software  complete 

m.  System  installation  complete 

n.  Start  test  and  evaluation 

o.  Test  and  evaluation  complete 

p.  Start  operations  evaluation 

q.  Operations  evaluation  complete 

r.  Deliver  to  Fleet 

s.  Project  complete. 

Appendix  7B  provides  the  format  for  detailed  milestone  reporting,  the  format  for  summary  milestone 
reporting,  and  the  instruction  for  preparation  and  completion  of  milestone  reports.  Milestone  reports 
should  be  monitored  and  maintained  on  a  continual  basis. 
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7.7  TECHNICAL  SEQUENCE  AND  FLOW— NETWORKS 


The  network  is  a  planning,  scheduling,  and  project  appraisal  document  which  may  be  used  as 
an  effective  management  control  tool.  Project  uncertainties  such  as  potential  slippages,  schedule  impact, 
problems  requiring  management  decisions,  and  decision  trade-offs  may  be  identified  by  the  use  of  the 
network.  A  typical  network  is  illustrated  in  Figure  7.1.  In  constructing  networks  as  a  planning  tool, 
the  following  two  principals  should  apply: 

a.  The  network  must  satisfy  the  needs  of  the  project  and  be  flexible  enough  to  include  any  project 
changes  which  may  occur. 

b.  The  network  must  be  comprehensive  to  the  user,  provide  timely  information,  and  be  worth 
the  cost  of  development  and  maintenance. 

The  network  should  contain  the  following  elements: 

a.  Activity — an  incremental  element  of  the  network  that  defines  a  specific  effort  to  be  accomplished 
over  a  period  of  time.  The  activity  is  usually  portrayed  as  a  line  identified  by  the  activity  title. 

b.  Event — a  point  in  time  on  a  network  usually  portrayed  as  a  circle  at  the  beginning  and  end 
of  an  activity.  The  event  may  be  identified  by  a  unique  number. 

c.  Time — elements  of  time  assigned  to  each  activity  usually  portrayed  in  decimal  increments,  i.e. , 
1.0  equals  1  week,  2.0  equals  2  weeks,  etc. 

d.  Critical  path — the  longest  path  or  controlling  chain  of  activities  within  a  network  in  which  any 
slippage  will  delay  the  network  end  event. 

Network  logic  display  represents  the  planned  flow  of  activities  and  constraints  as  known  or  perceived 
by  the  personnel  responsible  for  performance.  Identification  of  activities  and  constraints  are  best  defined 
during  integrated  planning  sessions.  Project  managers  should  participate  in  the  integrated  planning 
sessions  to  establish  network  logic.  Time  spans  are  assigned  to  each  activity  when  it  is  established  that 
the  network  logic  is  a  responsible  portrayal  of  the  activities  to  be  accomplished. 

Latest  allowable  dates  (TL)  are  determined  by  subtracting  activity  time  spans  from  the  ending 
event  date.  The  longest  path  determined  is  the  critical  path. 

Earliest  expected  completion  dates  (Te)  are  determined  by  adding  time  spans  of  uncompleted 
activities  from  time  now  through  the  ending  activity. 

Slack  is  determined  by  subtracting  the  earliest  expected  completion  date  from  the  latest  allowable 
date  (TL-Te).  The  resultant  positive  or  negative  slack  determines  the  critical  path. 

The  earliest  expected  date  (Te)  may  be  used  to  designate  interim  milestones. 


7.8  WORK  BREAKDOWN  STRUCTURE 

The  work  breakdown  structure  (WBS)  is  defined  as  "a  product-  or  task-oriented  document  which 
depicts  the  end  item  of  a  project,  its  subdivisions,  interrelationships,  and  levels  of  detail.”  A  controllable 
unit  of  work  or  dement  in  a  WBS  is  termed  a  work  package.  The  primary  function  of  the  WBS  is 
to  provide  a  systematic,  end  item-oriented  breakdown  of  hardware,  software,  services,  and  other  work 
packages  that  are  required  to  make  up  the  total  system.  Indentures  in  the  WBS  are  defined  as  work 
package  levels,  i.e.,  the  first  level  is  the  system  end  item,  the  second  level  consists  of  work  package 
segments  of  the  end  item,  each  of  which  may  be  further  segmented.  Each  work  package  is  identified 
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figure  7.1.  Technical  sequence  a  id  flow  network  activity  oriented. 


T*V 


by  an  element  number.  WBS’s  should  be  built  to  the  level  of  detail  that  most  significantly  depicts  the 
work  to  be  performed.  The  lowest  level  of  detail,  however,  should  be  a  controllable  level  in  areas  of 
performance,  schedule,  and  cost.  Graphic  and  indenture  examples  of  WBSs  are  provided  in  Figures 
7.2  and  7.3,  respectively.  Preparation  of  the  WBS  is  governed  by  MIL-STD-881A.  After  proper  input 
procedures  have  been  followed,  both  figures  may  be  obtained  through  the  automated  WBS  system. 
The  WBS  is  a  valuable  tool  in  the  development  of  projects,  systems,  equipment,  or  other  material  and 
service  items.  Its  use  is  indicated  for  any  project  which  can  be  broken  down  into  a  significant  number 
of  controllable  work  packages.  The  WBS  is  easily  prepared  manually  for  smaller,  short-lived  projects. 
For  larger,  longer  projects,  automation  is  available  and  recommended. 

7.8.1  Automated  Work  Breakdown  Structure  System 

To  facilitate  WBS  reporting  requirements,  an  automated  WBS  generation  system  is  available.  In 
addition  to  the  planning  function  of  the  WBS,  machine-generated  reports  are  available  to  provide  the 
project  manager  with  timely  information  (on  an  as-required  basis)  of  the  cost  estimates  versus  to-date 
charges  and  obligations  at  all  levels  of  the  structure. 

The  following  reports  in  WBS  format  are  available: 

a.  Report  WBS1  includes  all  levels  of  the  structure.  There  are  three  data  format  options  for  this 
report  (Appendix  7C): 


Option  1 

Estimates 

*  (YTD)  Charges 
***  (MTD)  Charges 
Balance 


Option  2 

Estimates 

*  (YTD)  Costs 
***  (MTD)  Costs 
Encumbrance 
Balance 


Option  3 

Estimates 

**  (ITD)  Costs 
•••  (MTD)  Costs 
Encumbrance 
Balance 


*  Year  to  Date 
**  Inception  to  Date 
***  Month  to  Date 

Each  of  the  above  reports  conclude  with  a  recap  by  project  number  as  shown  in  Appendix  7D. 

b.  Report  WBS2  is  the  same  as  above  except  this  structure  only  shows  the  first  three  summary 
levels  (levels  1,  2,  and  3).  See  Appendix  7E. 

c.  Report  WBS3  is  a  heavily  indented  report  showing  each  element  number  within  the  W'BS,  along 
with  the  total  estimate  of  each  work  package  and  the  WBS  work  package  level.  See  Appendix  7F. 

To  initiate  a  WBS  within  the  automated  work  breakdown  structure  system,  the  project  manager 
should  direct  the  preparation  of  the  WBS,  ensuring  that  the  WBS  worksheet  shown  in  Appendix  7G 
is  prepared  in  accordance  with  the  instructions  included  with  it.  Completed  forms,  along  with  instructions 
as  to  which  report  options  and  what  reporting  frequency  is  desired,  may  be  submitted  to  data  processing 
for  action. 

A  WBS  element  number  should  be  established  for  each  work  package  of  the  WBS.  Figure  7.3 
provides  two  options  that  may  be  used  in  structuring  a  WBS  element  numbering  system. 
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Figure  7.2.  Work  breakdown  structure  (graphic  example). 


Work  breakdown  structure  element  numbering  system. 
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7.8.2  WBS  Element  Description  Record 

In  conjunction  with  the  WBS,  a  WBS  Element  Description  Record  as  formatted  in  Appendix  7H 
will  be  filled  out  for  each  element  to  a  practical  level  in  the  WBS.  Instructions  for  preparing  this  document 
are  also  included.  These  records  will  be  maintained  as  an  integral  part  of  project  plans. 


7.9  PROJECT  PLANNING  RANGE 

Adherence  to  directives  is  a  critical  part  of  project  management.  To  this  end,  a  project  management 
questionnaire  has  been  developed  to  aid  the  project  manager  in  determining  a  broad  range  ot  directive 
subjects  which  require  consideration  during  the  project  development  phase.  The  questionnaire  covers 
such  subjects  as  project  authorizations,  project  management,  project  documentation,  procurements, 
systems  engineering,  data  management  integrated  logistics  support,  test  and  evaluation,  security,  and 
safety.  The  project  management  questionnaire  as  delineated  in  Appendix  71  is  provided  as  a  guide  to 
the  management  decision  process. 


7.10  SCHEDULES 


7.10.1  Bar  Charts 

Bar  charts  are  used  as  a  visual  indicator  depicting  size  variations  when  compared  to  a  given  scale. 
They  provide  a  relatively  simple  visual  aid  to  the  project  manager  and  function  as  a  management  tool 
for  communicating  status  and/or  decisionmaking. 

7.10.2  Gantt  Charts 

Gantt  charts  portray  performance  or  output  related  to  time  and  are  commonly  used  as  master 
schedules,  progress  charts,  and  milestone  charts.  Gantt  charts  were  first  developed  by  Henry  L.  Gantt 
early  in  the  twentieth  century.  Since  then,  many  variations  have  been  used  successfully.  The  primary 
purpose  of  Gantt  charts  is  to  convey  schedule  conditions  and  status  to  the  manager.  They  will  also 
be  used  to  track  actual  completion  data. 

A  series  of  activities  is  portrayed  on  a  Gantt  chart  by  the  use  of  horizontal  bars  under  a  dateline. 
The  area  under  a  dateline  is  commonly  referred  to  as  the  plotting  plan.  Bars  or  lines  may  be  color 
coded  to  signify  status  and/or  progress.  Color  coding  provides  a  ready  means  for  the  manager  to 
determine  the  status  of  ongoing  tasks. 

Constraints  are  portrayed  on  the  plotting  plan  by  the  use  of  vertical  dash  lines.  Arrows  are  used 
to  signify  direction  of  constraints.  Dash  lines  used  in  a  horizontal  direction  signify  unscheduled  time. 
Specific  points  in  time  or  events  are  indicated  by  the  use  of  small  symbols  such  as  triangles,  squares, 
etc.  Legends  are  placed  just  below  and  to  the  right  of  the  plotting  plan. 

Figure  7.4  is  provided  as  an  example  master  schedule  illustrating  the  Gantt  chart  concept.  Schedule 
sequence,  status,  and  progress  should  be  portrayed  in  such  a  manner  that  it  is  readily  understood  by 
the  reader. 


7-13 


JAN  Ff»  MAR  I  APR 


Figure  7.4.  Master  schedule  (Gantt  chart  example) 


7.10.3  Line  Charts 


Line  charts  plot  the  movement  of  one  or  more  quantities  over  a  period  of  time.  Time  units  are 
shown  horizontally  and  quantities  are  shown  vertically.  Schedule  versus  actual  information  may  be 
measured  by  plotting  actual  data  at  specific  points  in  time  during  the  life  of  the  schedule. 


7.11  PROJECT  PROGRESS  REPORTS 

7.11.1  General 

Progress  reports  have  been  prepared  and  submitted  to  sponsors  in  many  different  formats  over 
the  past  years.  Appendix  7J  provides  a  project  progress  report  format.  Status  that  is  pertinent  to  overall 
project  management  and  of  specific  interest  to  the  sponsor  is  reflected  in  the  progress  report.  Conveyance 
of  internal  details  and/or  milestones  that  are  not  significant  are  to  be  avoided. 

7.11.2  Objectives 

The  standard  progress  report  format  is  designed  to  provide  a  tool  for  project  reviews  and  assessment. 
To  this  end  the  project  progress  report: 

a.  Provides  for  continuity  of  status  reporting 

b.  Provides  a  better  understanding  of  project  status,  progressions,  problems,  and  planned  corrective 
actions 

c.  Facilitates  rapid  and  understandable  review  by  higher  management 

d.  Provides  a  common  outline  for  historical  cataloging 

e.  Allows  for  comparative  analysis  data  within  and  between  projects. 

7.11.3  Procedure 

Reporting  of  financial  data  through  the  project  progress  report  is  contingent  upon  specific 
requirements  of  both  the  project  manager  and  the  sponsor.  Appendix  7J  also  provides  the  instruction 
for  preparation  and  completion  of  the  progress  report. 


7.12  PROCUREMENT  PLANNING 
7.12.1  Contract/Purchase  Order  Planning 

Procurement  planning  is  defined  as  a  series  of  decisions  directed  to  the  integration  of  procurement 
with  technical  and  financial  plans  during  the  acquisition  cycle.  The  planning  process  must  be  performed 
well  in  advance  of  procurement  initiation  in  order  to  foresee  potential  procurement  problems.  The 
planning  process  applies  to  all  major  and  minor  procurements.  Several  directives  are  available  with 
regards  to  procurement  planning.  They  are  SECNAVINST  4200.31,  NAVMATINST  4200. 30D, 
NAVMATINST  4200.49,  NAVMATINST  4200. 50C,  NAVMA  f  INST  4200.52,  NAVMATINS  I 
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4200.54,  NAVMATINST  4200.55,  OPNAVINST  5000.42B,  SPAWAR1NST  4200.6D,  NAVMATINST 
5000.29A,  and  FAR  Section  1,  Part  21,  Appendix  J. 

The  responsibility  for  procurement  planning  lies  with  the  project  manager.  The  degree  of  planning 
required  for  procurements  depends  to  a  great  extent  on  the  dollar  amount  of  the  procurement.  The 
Contract  Division,  Code  21,  is  responsible  for  providing  expertise  in  the  area  of  procurements  to  all 
of  the  technical,  scientific,  and  management  personnel  at  NOSC.  The  project  manager  has  the 
responsibility  for  reviewing  and  taking  responsibility  for  procurement  requirements  being  processed 
in  support  of  the  project.  Code  211,  the  Purchase  Branch,  is  responsible  for  processing  authorized 
procurements  up  to  $5,000.  The  contract  branches  have  authority  for  processing  procurements  for  $5,000 
or  more. 

The  planning  of  procurements  is  interrelated  with  early  requirement  definition  accomplished  during 
the  exploratory  and  advanced  development  stages  of  system  development.  Determination  of  the  hardware 
and  software  requirements  of  a  system  is  an  integral  part  of  the  system  engineering  process.  Plans  for 
procurement  actions  should  be  formed  during  development  of  functional  and  systems  specifications. 

The  schedule  impact  of  long  lead  time  major  procurements  is  of  primary  concern  to  the  project 
manager.  To  this  end,  project  managers  should  ensure  an  in-depth  study  of  the  approaches  and 
requirements  for  acquisition  plans  (APs). 

Acquisition  planning  is  necessary  for  all  procurements  at  the  project  level.  The  process  may  be 
defined  as  a  series  of  decisions  directed  to  the  integration  of  procurement,  technical,  and  financial 
plans  during  the  project/weapon  systems  acquisition  cycle. 

The  goal  of  advanced  procurement  planning  is  to  obtain  a  successful  system,  in  a  timely  manner, 
at  the  lowest  total  cost  to  the  Navy.  Realistic  milestones  with  adequate  lead  times  should  be  depicted 
on  a  graphic  timetable  display.  This  scheduling  action  should  be  accomplished  well  in  advance  of 
procurement  package  preparation.  Procurement  milestones  should  be  interrelated  as  part  of  all  other 
activities  of  the  system  development  in  order  to  determine  the  most  critical  areas  with  regards  to  lead 
time.  Graphic  displays,  such  as  the  Gantt  chart  in  Figure  7.4,  may  be  used  in  the  procurement  planning 
process. 

7.12.2  Acquisition  Plan  Requirements 

The  AP  is  a  formal  document  that  is  prepared  for  the  purpose  of  defining  major  procurement 
programs  during  the  life  cycle  of  systems  development.  FAR,  Part  7,  addresses  acquisition  plans.  The 
document  in  part  states: 

a.  “Navy  contracting  activities  shall  perform  coordinated  planning  as  prescribed  by  FAR,  Subpart 
7.1,  for  those  acquisitions  which  meet  the  criteria  and  thresholds  in  FAR  7.103(a)(1).  The 
acquisition  plan  required  by  FAR,  Part  7,  will  be  the  principle  document  for  in-depth  program 
review  and  oversight  by  the  Navy  acquisition  executives  who  are:  (1)  ASN  (RE&S)  for  R&D, 
(2)  ASN  (FM)  for  ADP,  and  (3)  ASN  (S&L)  for  other  procurements  (the  R&D  funded  portions 
of  ship  design  and/or  construction  procurements  are  under  the  joint  cognizance  of  ASN  (RE&S) 
and  ASN  (S&L)).” 

b.  A  program  endorsement  memorandum  (PEM)  must  be  obtained  from  the  cognizant  Navy 
acquisition  executive  for  the  following  APs  (when  DFARS/NARSUP  7. 103(c)(2)  requires  that 
a  written  AP  be  prepared): 

(1)  “Development  acquisitions  (see  FAR  35  001),  other  than  those  funded  under  program 
element  6.2,  whose  total  contractual  cost  is  estimated  as  greater  than  S5  million  for  NAVAIR 
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and  NAVSEA  and  $2  million  for  all  others.” 

(2)  “All  shipbuilding  programs.” 

(3)  “All  production  programs,  regardless  of  type  of  funds  to  be  utilized  or  DON  contracting 
activity  performing  the  procurement,  having  a  single  contract  value  whose  value  (including 
options)  is  estimated  to  equal  or  exceed  $50  million.” 

FAR  7. 103-90  provides  for  the  acquisition  plan  format.  Every  topic  may  not  be  applicable  to  every 
acquisition;  however,  the  format  can  serve  as  a  checklist  of  material  to  be  included  in  the  AP.  Cross 
references  have  been  made  to  FAR,  DFARS,  and  MARSUP  requirements  for  acquisition  plan  content. 
Appendix  7K  provides  an  outline  of  the  plan  (plus  a  list  of  required  tables)  to  project  management 
for  ready  reference.  Acquisition  plans  for  many  projects  may  have  been  prepared  by  the  sponsoring 
activity  or  at  the  applicable  SYSCON  level.  Procurement  requirements  that  may  require  a  NOSC-initiated 
acquisition  plan  should  be  coordinated  through  the  Supply  Department  to  ensure  that  the  necessity 
for  an  acquisition  plan  exists  and  to  ensure  that  proper  procedures  are  being  followed. 


Appendix  7A 


Model  Project  Management  Plan 
(See  subsection  7.4) 
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SECTION  1 


INTRODUCTION 


1 . 1  PURPOSE 


The  purpose  of 
approach  for  the 
resources  that  will 


this  Project  Management  Plan  is  to  present  the  development 
—  (project)  _  and  to  define  the  management  tools  and 


be  utilized.  The  plan  is  logically  divided  into  six 


sections  covering  all  aspects  of  NOSC  project  management: 


Section 


Title 


1 

2 

3 

4 

5 

6 


Introduction 

Approach 

Project  Management 
Milestones  and  Schedules 
Financial  Plan 
Work  Breakdown  Structure 


1.2  TASK  ASSIGNMENT 


The 


(project) 


(date ) 


was  established  in  response  to  (RCO,  etc.)  on 


- .  The  essential  elements  of  the  Project  are  to  conduct 

design,  development,  teest  and  evaluation  of  _ (project) _ . 


(project) 


is 


The  Principal  Development  Activity  (PDA)  for 

-  ^gponsor ) - .  The  Naval  Ocean  Systems  Center  (NAVOCEANSYSCEN) ,  Sar 

Diego,  California,  has  been  designated  the  Performing  Activity  (PA)  bj 
-  (sponsor ) _  task  assignment  letter  _ (date) 


1 . 3  PROBLEM  DEFINITION  AND  BACKGROUND 


Describe  the  problem  and  circumstances  that  led  to  the  requirement  for 
this  project.  Also,  generally  define  the  historical  background. 


1.4  OBJECTIVES 


Define  the  objectives  of  the  project  and  the  expected  results  of  the 
effort.  (The  objectives  may  be  to  develop  a  feasibility  model  and  then  an 
advanced  development  model  (ADM)  which  will  finally  lead  to  an  engineering 
development  model  (EDM)). 

1 . 5  GENERAL  SYSTEM  DESCRIPTION 

In  general  terms,  supported  by  a  block  diagram,  describe  the  system. 
(One  or  two  paragraphs). 


SECTION  2 


APPROACH 

2.1  GENERAL 

This  section  defines  the  approach  to  be  taken  in  solving  the  problem 
defined  in  paragraph  1.3.  It  includes  the  development  approach  which  covers 
such  areas  as  planning,  design,  fabrication,  and  test  and  evaluation.  This  is 
then  followed  by  the  technical  approach  which  goes  more  into  the  technical 
detai  Is . 

2 . 2  DEVELOPMENT  APPROACH 

Explain  who  will  develop  each  objective  (ADM/EDM),  what  tests  will  be 
performed,  including  use  of  COMOPTEVFOR  and/or  other  agencies.  Also  include 
development  of  specifications  and  the  procurement  packages.  A  general  tech¬ 
nical  sequence  and  flow  diagram  may  be  used  to  support  the  text;  e.g., 


Research-Exploratory 

Advanced  Development 

Engineering 

Development 

Development 

Sys.  Dev.  ^ 

V 

_ y 

Sys.  Engr. 

V  K7 

\7 

Documentation 

v_ 

V  V 

_sz 

Software 

^7 

T&E 

_ V 

This  section  may  be  expanded  to  include  a  brief  description  of  the  proposed 
technical  approach. 


2.2.1  Pre-Task  Planning 


The  detailed  planning  and  definition  of  tasks  and  responsibilities  shall 
be  accomplished  as  follows: 

a.  Summary  Work  Breakdown  Structure  (WBS)  should  be  prepared  and 
included  in  Section  6.  In  conjunction  with  the  preparation  of  this  WBS,  WBS 
Element  Description  Records  should  be  prepared  and  maintained  as  an  integral 
part  of  this  plan.  These  records  shall  be  used  for  negotiation  with  the  re¬ 
sponsible  activities  to  finalize  the  detailed  tasks  and  responsibilities  to  be 
delegated . 


b.  Contractual  specifications  and  procurement  package  data  identified  by 
the  finalized  EM  definition  shall  be  prepared  prior  to  entering  the  design  and 
fabrication  phase. 

c.  Test  planning  shall  be  implemented  relevant  to  technical  evaluation 
requirements . 

2.2.2  Design  and  Fabrication 

Factors  to  be  considered  in  the  EM  design  are  addressed  in  the  EM  defini¬ 
tion.  Design  development  shall  be  guided  by  continuing  in-house  design 
reviews  and  by  formal  design  reviews  as  scheduled  in  Section  4.  As  occurring 
design  reviews,  design  requirements  shall  be  modified  on  the  basis  of  updated 
evaluations  or  changes  in  relative  criticality  disclosed  during  these  reviews. 

Quality  assurance  shall  be  obtained  by  means  of  inspection  and  testing  of 
components  and  subassemblies  prior  to  assembly  integration. 

2.2.3  Test  and  Evaluation 

Tests  and  evaluations  are  to  be  conducted  as  reflected  in  the  WBS.  Test 
plans  and  procedures,  with  subsequent  test  results,  shall  be  sunmitteu  lor' 


integration  into  the  overall  support  test  plan  and  engineering  test  report  as 
appropriate . 


2.2.4  Preliminary  EDM  Procurement  Preparation 

To  provide  a  smooth  transition  for  entering  into  engineering  development, 
data  evolving  from  each  event  in  the  scheduled  program  shall  be  collected  by 
the  project.  These  data  shall  be  used  in  the  preparation  of  descriptions  of 
services  and  materials  for  procurement  packages  to  be  used  in  soliciting 
invitations  for  bids  in  competitive  procurement. 

2.3  TECHNICAL  APPROACH 


Write  a  comprehensive  narration  of  the  system  and  the  technical  approach 
as  to  development  of  the  system. 
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SECTION  3 


PROJECT  MANAGEMENT 


3.1  GENERAL 

The  Principal  Development  Activity  (PDA)  for  (project)  is  (sponsor). 
NAVOCEANS YSCEN  is  the  Performing  Activity  for  development  and  management  of 
the  (project).  Specific  responsibility  in  fulfillment  of  project  objectives 
are  assigned  to  NAVOCEANS  YSCEN  by  (sponsor)  in  the  form  of  (refer  to  sponsor 
authorization).  This  section  identifies  the  organizational  structure  and 
management  controls  employed  in  fulfillment  of  the  objective. 


3 . 2  ORGANIZATION 

Figure  3-1  depicts  the  relationships  among  organizations  involved  in  the 
(project).  NAVOCEANS YSCEN,  Code  (managing  code)  is  the  (project)  and  is 
responsible  for  project  management.  NAVOCEANSYSCEN  technology  codes  will 
accomplish  specified  work  in  support  of  the  (project)  development  under  spe¬ 
cific  work  assignments.  Other  activities  and  contractors  will  be  tasked  as 
required. 

3.3  MANAGEMENT  CONTROL  SYSTEM 

Management  controls  will  be  exercised  by  the  (project)  over  task  assign¬ 
ments,  manpower  expenditures,  cost,  and  scheduling.  These  controls  will  be 
keyed  to  specific  elements  of  a  Work  Breakdown  Structure  (WBS)  constituting 
the  bases  for  task  identification  and  baseline  configuration  identification. 
The  WBS  is  described  in  Section  6. 

3.3.1  Task  Assignments 

Tasks/Work  packages  as  identified  on  the  Wfas  Clement  Description  Record 
will  be  assigned  to  appropriate  Project  Managers.  Manpower  and  funding  will 
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Figure  3-1 

Organizational  Chart 


be  allocated  to  these  tasks  for  subsequent  correlation  with  expenditure  rates 
tabulated  by  the  cognizant  NA VOCEANS YSCEN  Project. 

3.3.2  Cost  Control 

The  plannned  expenditure  rate  for  (project)  is  shown  in  Figure  3-3  for  FY 

( _ )  •  Cumulative  costs  identified  by  weekly  MIS  printouts  for  each  task 

will  be  correlated  with  the  projection.  Using  this  method,  the  effects  of 
major  program  expenditures  and  unexpected  trends  in  the  expenditure  rate  will 
be  readily  determined,  permitting  program  adjustments  or  corrective  action  to 
be  taken.  The  financial  plan  by  WBS  element  is  given  in  Section  5. 


Figure  3-3  Expenditure  Profile 

3.3.3  Manpower  Expenditure 

The  cumulative  total  manhours  expended  against  each  activity  and/or  work 
package  will  be  tabulated.  The  rate  of  expenditure  will  be  correlated  with 
the  anticipated  rate  shown  on  the  Manpower  Loading  Chart  in  Table  3-1 . 

3.3.4  Milestone  Control 

The  (project)  milestones  will  be  monitored  and  controlled  by  periodic 
reviews  of  the  milestone  lists  in  Section  4.  Progress  for  each  activity  will 
be  correlated  to  these  lists  permitting  corrective  action  to  be  taken  as 
required. 

3.3.5  Design  Reviews 

Design  reviews  will  be  held  periodically  with  all  participating  project 
organizations  and  NAVOCEANSYSCEN  technical  codes.  The  reviews  are  significant 
as  a  medium  of  assuring  technical  compatibility  at  component  interfaces. 
Design  reviews  direct  management  attention  to  the  status  of  accomplishment  and 
significant  technical  or  management  problems.  System  Preliminary  Design  Re¬ 
views  ( PDRs )  and  Critical  Design  Reviews  (CDRs)  will  be  held  as  required,  with 
all  cognizant  participants  and  the  sponsor. 
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Table  3-1.  Manpower  Loading 


3.4  MANAGEMENT  REPORTS 

3.4.1  Monthly  Reports 

Monthly  reports  of  status  will  be  submitted  to  (sponsor)  on  appropriate 
NAVOCEANSYSCEN  project  progress  report  formats.  The  reports  will  include  the 
status  of  milestone  accomplishment,  the  status  of  funds  and  significant  events 
occurring  during  the  month. 

3.4.2  Conference  Reports 

Narrative  reports  will  be  submitted  of  formal  and  informal  conferences 
with  staff  personnel  or  other  commands. 

3.4.3  Design  Review  Reports 

Reports  of  Design  Review  Meetings  will  be  made  and  furnished  to  (sponsor) 
and  all  supporting  project  organizations  and/or  NAVOCEANSYSCEN  codes. 


3 . 5  SYSTEM  ENGINEERING 

3.5.1  System  Definition 

The  primary  tasks  of  system  engineering  during  Advanced  Development  in¬ 
volve  derivation  of  an  optimum  equipment  configuration  and  software  control 
mechanisms  that  will  achieve  the  capability  of  the  system  to  search,  acquire, 
track  and  communicate.  The  basic  system  components  and  functional  area  per¬ 
formance  requirements  have  been  defined  and  are  documented  in  the  Type  A 
System  Specification.  This  phase  of  system  engineering  will  culminate  with 
preliminary  test  results  upon  completion  of  subsystem  bench  tests. 

3.5.2  System  Optimization 

As  system  testing  progresses  from  bench  tests  through  final  testing,  data 
will  be  obtained  that  will  prove  the  functional  design  integrity  or  indicate 
areas  where  trade-offs  are  necessary.  Reconfiguration  studies  will  be 
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conducted  as  appropriate  during  the  period  following  these  tests.  The  results 
will  permit  finalization  of  the  system  and  critical  item  specifications  for  4*5*. 

the  RDM  procurement  package.  During  Engineering  Development,  the  system 
engineering  function  will  be  expanded  to  include  all  aspects  of  reliability, 
maintainability  and  support  required  by  the  operational  configuration. 

3.6  CONFIGURATION  MANAGEMENT 

Formal  configuration  management  will  be  implemented  at  the  beginning  of 
the  Engineering  Development  Phase.  It  is  desirable  to  maintain  design 
flexibility  until  the  optimum  configuration  has  been  established.  Appropriate 
control  will  be  exercised  by  the  (project)  to  assure  adequate  documentation  of 
the  design  and  translation  of  pertinent  data  to  the  final  system  and  critical 
item  specifications  for  the  EDM.  Planning  for  configuration  management  will 
be  undertaken  concurrent  with  the  preparation  of  the  EDM  procurement  package. 


SECTION  4 


MILESTONES  AND  SCHEDULES 


4.1  GENERAL 

The  milestones  and  schedules  for  the  (project)  will  be  included  in  this 
section?  each  milestone  should  be  keyed  to  the  related  WBS  element  number. 
(See  Section  6). 

4.2  PROJECT  MILESTONES  AND  SCHEDULES 

Development  of  the  (project)  involves  accomplishment  of  specific  tasks 
which  are  sometimes  dependent  on  completion  of  previous  tasks.  The  major 
tasks,  their  relationship  to  each  other,  and  the  schedule  should  be  shown  in 
Figure  4-1 ,  Time  Dependency  Chart.  All  major  milestones  should  also  be 
identified  on  this  chart.  (Table  4—1  is  a  list  of  the  (project)  milestones 
and  Table  4-2  is  a  list  of  deliverables.) 


EXAMPLE 


Feasibility  Study 
(WBS  1.1.3) 


Checkout 
(WBS  3.4.2) 


Time 


Design  j 

(WBS  1.1.6)  | 


\  Sys.  Engr 


(WBS  6.1.1) 


Prepare  a  flow  chart  showing  dependencies,  WBS,  and  task 
descriptions.  This  chart  may  be  a  large  foldout  page. 


Figure  4-1.  Time  Dependency  Chart 

NOTE:  This  should  be  one  of  the  first  tasks  accomplished  by  the 
Project  Manager. 


Figure  4-1.  Milestones 


|  Related  | 

|  WBS  |  Milestone 


|  Date  | 


Table  4-2  List  of  Deliverables 


ELEMENT 

TYPE  OF 

DATE  OF 

NUMBER 

DELIVERABLE 

DELIVERABLE 

DELIVERABLE 

7.2.7 

EM  Definition 

Preliminary 

6.1.2 

Program  Plan 

Fi  nal 

• 

(N 

• 

r~ 

EM  Definition 

Final 

7.2.7 

EM  Development  Support  Test 

Final 

Plan 

5.5.1 

Installation  Plan 

Final 

6.1.2- 

7.3 

Program  Plan  (Updated) 

Final 

7.2.5 

System  Engineering  Test 

Final 

Report 

7.2.6 

EDM  Spec  and  Data  Package 

Final 

EXAMPLE 
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SECTION  5 


FINANCIAL  PLAN 


5.1  FINANCIAL  REQUIREMENTS 

The  total  cost  for  _ (project) _  is  shown  in  Table  5-1.  Cost  for 

major  activities  and  lower  work  elements  (level  3)  for  each  FY  is  shown  in 
Table  5-2.  Labor  and  overhead,  major  procurement,  material  and  travel  costs 
from  the  individual  tasks  (level  4)  identified  in  the  Work  Breakdown  Structure 
of  Section  6  were  utilized  in  the  preparation  of  these  cost  summary  figures. 

Major  procurement  and  material  costs  can  be  summarized  by  subsystem  in 
subsequent  tables.  Corresponding  level  3  Work  Breakdown  Structure  line  item 
numbers  can  be  shorn  in  the  major  procurement  and  material  summary  tables  to 
permit  cross  reference  to  the  corresponding  tasks  in  the  WBS. 

5.2  COST  CONTROL 


The  NAVOCEANSYSCEN  Management  Information  System  (MIS)  will  be  used  to 
report  and  document  cost  control.  This  MIS  consists  of  computerized  reports 
of  all  expenditure  (labor,  material,  and  travel)  and  will  be  delivered  to  the 
project  manager  on  a  weekly  basis.  The  reports  will  be  keyed  to  the  WBS  (see 
detail  Section  6).  The  project  manager  will  correlate  technical  progress  to 
project  milestones  and  financial  status  on  a  regular  basis  to  assess  total 
project  status.  With  this  information,  the  project  manager  will  exercise 
effective  cost  control  of  the  project. 


5.3  FINANCIAL  REPORTING 

A  summary  of  project/subsystem  financial  status  will  be  delivered  to  the 
program  manager  (sponsor)  on  a  quarterly  basis. 
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SECTION  6 


WORK  BREAKDOWN  STRUCTURE 


6.1  INTRODUCTION 

The  major  (project)  management-control  tool  will  be  the  work  breakdown 
structure  (WBS).  The  WBS  is  a  product-oriented  device  composed  of  hardware, 
software,  services,  and  other  work  tasks  which  result  from  project  engineering 
efforts  during  the  development  and  production  of  a  project.  The  WBS  system 
enables  a  project  manager  to  logically  breakdown  the  overall  task  into  smaller 
workable  segments  that  can  individually  be  implemented,  managed,  and 
controlled  whereupon  completion,  the  total  task  will  be  accomplished.  Each 
task  in  this  system  is  assigned  a  WBS  element  number  which  can  be  controlled, 
by  ADP,  as  to  status,  costs,  and  recording. 

6.2  (PROJECT)  WORK  BREAKDOWN  STRUCTURE 

Figure  6-1  displays  the  product  to  be  developed  and  relates  the  elements 
of  work  to  be  accomplished  to  each  other  and  to  the  end  product.  Table  6-1  is 
an  indentured  form  of  the  WBS  which  enables  easy  revising  and  updating.  WBS 
Element  Description  Records  will  be  prepared  for  each  work  package  in  the  WBS, 
indicating  milestones,  deliverables,  and  responsibilities  for  that  particular 
work  package.  (See  Figure  6-2). 


! 
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Figure  6-1.  (Project)  Overall  WBS 


TABLE  6-1 


WORK  BREAKDOWN  STRUCTURE 


LEVEL 

1 

1 . 0  PROJECT 


LEVEL 

2 


LEVEL 

3 


LEVEL 

4 


2 . 0  SYSTEM  PROJECT 
MANAGEMENT 

2 . 1  PROJECT 
2 . 1  MANAGEMENT 


2.1.1  PROGRAM  PLAN 

2.1.2  CONFIG.  MANAGEMENT 

2.1.3  QUALITY  ASSURANCE 

2.1.4  COST  &  SKED  MGNT 

2.1.5  DATA  MANAGEMENT 

2.1.6  I.W.A,.'S  &  CONTRACT 

2.1.7  REPORTS  &  BRIEFING 

2.2  SYSTEM  ENGINEERING 

&  MANAGEMENT 


2.2.1  SYSTEM  EFFECTIVENESS 
(RMAILS) 

2.2.2  SPECIFICATIONS  (A&B) 

2.2.3  SYSTEMS  INTEGRATION 

2.2.4  SYSTEM  DESIGN 
ENGINEERING 

2.2.5  HUMAN  FACTORS 

2.2.6  SECURITY 

3.0  INJECTION  TERMINAL 
SUBSYSTEM 

3 . 1  INTEGRATION  & 

ASSEMBLY 

3.1.1  ASSY/SUB-ASSY 


EXAMPLE 


INTERFACES 

3.1.2  ELECTRICAL  INTE¬ 
GRATION 

3.1.3  MECHANICAL  INTE¬ 
GRATION 

3 . 2  COMPUTER  PROGRAMS 

3.2.1  EXECUTIVE  ROUTINES 

3.2.2  FUNCTION  ROUTINES 

3.2.3  DIAGNOSTIC  S>  MAIN¬ 
TENANCE  ROUTINES 

3.2.4  I/O  CONTROL  ROUTINES 

3.3  I/O  SUB-ASSEMBLIES 

3.3.1  MSG  ENTRY  DEVICE 

3.3.2  DISPLAY 

3.3.3  HARD  COPY  DEVICE 

3.3.4  CONTROL 

3.3.5  MMPS  MESSAGE  OUTPUT 

3.3.6  SELF-TEST 

3.4  ADP  SUB-ASSEMBLIES 

3.4.1  ARITHMETIC  SUB-ASSY 

3.4.2  MEMORY 

3.4.3  EXEC.  CONTROL 

3.4.5  I/O  CONTROL 
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WBS  ELEMENT  DESCRIPTION  RECORD 
11ND-NELC-3920/16  (1-73) 


|  ORIGINAL  DATE | REVISION  DATE | REVISION  LTR | SHEET  OF 

I  I  I  I  1 

_ I _ I _ I _ I _ 

WBS  ELEMENT  NO.  |  WBS  ELEMENT  TITLE 

I 

1.2. 3. 4  I  ILS  SUPPORT 


ENGINEERING  TASK  DESCRIPTION 


1.2. 3. 4  INTEGRATED  LOGISTICS  SYSTEM  (ILS)  SUPPORT 


Monitor  ILS  work  performed  under  contract.  Review  the  ILS 
plans  of  the  various  segment  and  subsystem  managers  for  compatibility 
with  the  over-all  program  ILS  plan. 

(1)  Prepare  an  analysis  of  the  T6E  Plans 

(2)  Provide  inputs  to  the  Configuration  Management  Plan, 
Project  Base  Line,  System  Specifications,  Technical  Interface 
Specifications,  Peripheral  System  Interface  Specifications,  T&E  Master 
Plan,  System  Test  Plan  and  COMSEC  Area  Plan. 


EXAMPLE 


Figure  6-2  WBS  Element  Description  Record  (1  of  2) 


1l  .U 


MILESTONE  REPORT 


MILESTONE  IDENTIFICATION 


MILESTONE  DATE  I  CHECK  ONE  I 


SCHEO  |  REVISED  [  ACTUAL 


|  unjUEIEEIS 


totals 


TAkENSANDLWHENjES  M'SSE°  "NCLU0E  REAS0NS-  EFFECT  ON  PROJECT.  REMEDIAL  ACTION 


14/ 


MILESTONE  REPORT 


SUMMARY 


FROM:  CODE: 


WEEK  ENDING 


mammal 


EXHIBIT  B-2 


(Reference  paragraph  5.0  of  the  course  syllabus) 


Instruction  for  the  Preparation  and  Completion 
of  Milestone  Reports 


Applicable  blocks  on  the  milestone  report  will  be  completed  as  follows: 


Block  Entry  Item 


Instructions 


1 .  From 

2.  To 

3.  Week  Ending 


Provide  the  code  of  the  cognizant  project 
manager ,  as  applicable. 

Provide  the  code  of  activity  that  the 
milestone  report  is  being  directed  to. 
Usually  this  is  the  project/division 
of f ice /sponsor . 

Enter  the  week  ending  date  covered  by  the 
milestone  report. 


4.  Project 


Enter  the  four  digit  project  number. 


5.  No.  Enter  sequential  numbers  as  required. 

This  field  serves  to  identify  the  mile¬ 
stone  by  a  single  reference  number. 


6. 


7. 


8. 


9. 


Milestone/Identification  Enter  the  milestone  names  listed  in 

sequence  by  earliest  date  first. 

Schedule  Enter  the  schedule  date  that  tne  mile¬ 

stone  is  to  be  accomplished. 

Revised  Enter  revised  dates,  if  applicable.  This 

block  will  be  used  only  when  scheduled 
milestone  dates  require  revision  in  con¬ 
sonance  with  project  objectives. 


Ac  tua 1 


Enter  the  actual  completion  of  accomp¬ 
lishment  date  of  the  milestone. 


1 0 .  Check  One  -  Made 


Enter  a  check  mark  if  milestone  was  made. 


1 1 .  Check  One  -  Missed 


Enter  a  check  mark  if  milestone  was  missed. 


Block 

Entry  Item 

Instructions 

12. 

Current  Week  -  Missed 

Enter  a  check  mark  if  milestone  missed 
uring  the  current  reporting  week. 

13. 

Totals 

Enter  the  total  count  of  check  marks  in 
block  columns  10,  11,  12. 

14. 

Discuss  Milestones  Missed 

Provide  reasons,  effects  on  project,  reme¬ 
dial  action  taken  or  to  be  taken,  and  when. 

15. 

Signature 

Project  manager  enters  signature. 

16. 

Date 

Self  explanatory. 

17. 

From 

Provide  the  code  number  from  which  the 
milestone  report  is  being  directed. 

18. 

To 

Provide  the  code  number  to  which  the  mile¬ 
stone  report  is  being  directed. 

19. 

Week  Ending 

Enter  the  week  ending  date  covered  by  the 
detail  milestone  reports. 

20. 

Project 

Enter  the  project. 

21. 

Project  Name 

Enter  the  name  of  the  project. 

22. 

Schedule 

Enter  the  number  of  milestones  scheduled. 

23. 

Made 

Enter  the  number  of  milestones  made. 

24. 

Missed 

Enter  the  number  of  milestones  missed. 

25. 

Current  Week  Missed 

Enter  the  number  of  milestones  missed  for 
the  current  week  only. 

26. 

Total 

Enter  totals  for  each  of  the  block  columns 
22,  23  and  24. 

27. 

Discuss  Milestones  Missed 

Provide  reasons,  effect  on  project,  reme¬ 
dial  action  taken  or  to  be  taken,  and  when. 

28. 

Signature 

Enter  authorized  signature. 

29. 

Date 

Self  explanatory. 
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Appendix  7C 


Report  WBS1  Options 
(See  subsection  7.8.1) 
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Report  WBS1  Recap 
(See  subsection  7.8.1 


Appendix  7E 
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Report  WBS2 
(See  subsection  7.8.1) 
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Instruction  for  the  Preparation  and  Completion 
of  Work  Breakdown  Structure  Worksheets 


Block 


1,  2 


Entry  Item 


WBS  Number 


3,  4 


Project  Title 


Level  1  Element  Title 


Element  Number 


Level  Number 


Element  Title 


Job  Order  Number 


Name 


Instruction 


Enter  an  arbitrary  number  to 
distinguish  this  WBS  from  any 
other.  This  number  may  be 
assigned  by  the  project  manager. 
Input  the  number  in  both  blocks 
a  and  2.  Fill  out  only  on 
page  1,  not  necessary  on  succeed¬ 
ing  pages. 

Enter  title  of  project  to  include 
two  lines.  The  title  will  be 
used  as  a  page  heading  on  all 
reports  for  this  WBS.  Each  line 
should  be  centered.  Fill  out  on 
page  1  only,  not  necessary  on 
succeeding  pages. 

Enter  the  title  of  level  1  (left 
justified) .  Fill  out  on  page  1 
only,  not  necessary  on  succeeding 
pages. 

Enter  the  element  number  (left 
justified) .  Input  with  decimal 
points,  i.e.,  1.4.13.6.3,  2. 3. 5. 2 
etc . 

Enter  level  of  this  element. 

Input  as  02  for  level  2,  06  for 
level  6,  etc. 

Enter  the  element  title  (left 
justified; . 

Input  a  job  order  number  for 
detail  elements  as  required.  If 
more  than  one  job  order  number  is 
associate  with  a  detail  element, 
input  a  duplicate  line  for  each' 
(left  justify) . 

Enter  the  name  of  the  person 
filling  out  forms. 


Block 


Entry  Item 


Instruction 


Code 


Extension 


Date 


Enter  the  code  of  person 
filling  out  forms. 

Enter  telephone  extension  of  per' 
son  filling  out  forms. 

Enter  the  date  forms  filled  out. 


Sheet 


Enter  the  sheet  number  and  total 
number  of  necessary  sheets,  in 
sequence . 


Structure  (WBS) 
Record 


7.8.2) 


WBS  ELEMENT  DESCRIPTION  RECORD 


MiTMirararo  b 

EM-IMM 1 

mmmmmm 

1  21  1 

WBS  ELEMENT  NO. 

WBS  ELEMENT  TITLE 

5/ 

6/ 

REVISION  LETTEI 
3/ 


ENGINEERING  TASK  DESCRIPTION 


/  •» 


Instruction  for  the  Preparation  of 
the  WBS  Element  Description  Record 


Block  Entry  Item  Instruction 


1  Original  Date  Enter  the  date  the  original  WBS 

element  was  completed. 

2  Revision  Date  Enter  the  date  each  time  the  WBS 

element  record  is  revised. 

3  Revision  Letter  Enter  a  revision  letter  each  time  the 

WBS  element  record  is  revised,  begin¬ 
ning  with  A  for  the  first  revision, 

B  for  the  second,  etc. 

4  Sheet  _  of  _  Number  of  WBS  element  record  sheets. 

5  WBS  Element  No.  Enter  the  assigned  WBS  element  number 

for  each  task,  i.e.,  l.l,  1.1.2, 

2.3,  etc.  Job  Order  number  may  also 
be  included  here  if  desired. 

6  WBS  Element  Title  Provide  a  short  descriptive  title  for 

WBS  element. 

7  Engineering  Task  Describe  each  task  in  detail  pro- 

Description  viding  a  of  description,  technical 

requirements . 


Project  Management  Questionnaire 


Prepared  by 

NOSC 
Code  805 
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PROJECT  MANAGEMENT  QUESTIONNAIRE 


I.  PROJECT  AUTHORIZATION 


Directive 


Sub j  ect 


Questions 


OPNAVINST 
5000. 42B 


II.  FUNDING 

SECNAVINST 
5000. IB 


SPAWAR 
4200. 6D 
FAR/DFARS 
Part  7.105 


Operational 
Requirements 
and  Objectives 


System 

Acquisition 


Acquisition 
Plan  (AP) 


III.  PROJECT  MANAGEMENT 


SECNAVINST 

5000.1 
3910.3 
5430.7 
5430.67 
5410.85 

DODDIR 

5100.1 

NAVMATINST 

3910.20 

NOSC  GUIDES 


NOSC  GUIDES 


NAVMATINST 
5000. 21A 
5000. 22A 


Organization 


Staffing 


Procedures 


Project 

Management 


1.  Has  project  authorization 
been  received? 


Has  adequate  funding  been 
received? 

Has  a  financial  plan  been 
prepared? 

Is  an  AP  required? 

Is  there  a  valid  AP? 

3.  What  is  AP  identifica 


Has  organization  structure 
been  established? 

Have  support  resources  been 
obtained? 


Is  manpower  level  adequate? 

Is  the  staff  trained  for 
project  requirements? 

Is  there  backup  for  key 
personnel. 

Is  there  a  current  func¬ 
tional  chart? 

Are  the  WBS  elements  defined? 

Who  is  handling  "public  infor¬ 
mation"  about  the  system? 


EXHIBIT  L 

(Ref  para  8.0  of  the  Course  Sylubus) 


page  1  of  10 


LlV.'.  «'-r  _ 
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_  rr*  a.* 
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NAVMAT 

P-9494 

NAVSO 

P-2457 


DODINST 

7000.2 


MIL-STD-881A 
NOSC  GUIDES 


NAVMATINST 
4855. 1A 

SELNAVINST 

4000.31 


Performance 

Measurement 


Program  Review 


Work  Breakdown 
Structure  (WBS) 


Quality 
Assurance  (QA) 


2.  Are  relevant  project  records 
available? 

3.  Who  is  responsible  for  liaison 
with  the  user? 

4.  Are  user's  requirements 
reviewed  for  possible 
improvement? 

5.  Is  someone  tracking  progress 
of  other  closely  related  sys¬ 
tems  that  could  affect  this 
system? 

1.  If  project  exceeds  $25M  of 

RDT&E  funds  or  $100M  of  cumu¬ 
lative  project  investment,  is 
DODINST  7000.2  followed? 

1.  Does  project  review  provide 
coverage  for  cost,  schedule 
accomplishment,  technical 
performance,  and  logistic 
support? 


1.  Has  a  WBS  been  prepared? 

2.  Does  it  include  all  task 
descriptions? 

3.  Have  provisions  been  made  for 
updating? 

1.  Have  the  elements  of  QA  been 
considered  for  the  material 
cycle,  i.e.,  contract 
definition,  engineering 
development,  production  and 
service  life? 


2.  Has  life  cycle  costing  been 
considered? 


IV .  PROJECT  DOCUMENTATION 


NAVMATINST 
5200. 11B 


System 

Description 


1.  Is  there  a  system  description 
written  (non-technical)  ? 


Project  Master 
Plan  ( PMP) 


1.  Is  a  PMP  required? 

2.  If  PMP  is  NOT  required,  is  an 
alternate  prepared? 

3 .  Has  it  been  approved  ? 

4.  Does  it  cover: 

Historical  data 
Current  requirements/ 
ob j  ectives 
Summary  highlights 
System  description 
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Project  management  plan 
Project  milestone  plan 
Definition  plan 
Development,  test,  and 

evaluation  plan  ^ 

Production,  installation 
and  base  loading  plan 
ILS  plan 

Personnel  and  training  plan 
System  effectiveness  plan 
Reliability/maintainability 
plan? 


1.  Are  technical  manuals  (hard¬ 
ware/software)  required? 

2 .  Will  they  be  contracted  or 
done  in-house? 

3.  Have  outlines  been  prepared 
and  approved? 

4.  Has  a  valid  verification 
plan  been  prepared? 

5.  Have  the  manuals  been 
validated/verified? 

1.  Do  progress  reports  cover: 

Complete  status 
Potential  problems 
Financial  status 
Milestone  report/ status 
Critical  problems? 


MIL-STD-490 

Specifications 

1. 

Has  a  system  spec  (Type  A) 
outline  been  developed  and 

approved? 

2. 

Has  the  system  spec  been 
developed? 

3. 

Have  development  specs 
(Type  B)  been  developed? 

4. 

Have  product  specs  (Type  C) 
been  developed? 

SECNAVINST 

Software 

1. 

Have  the  software  specs  been 

5233. IB 

developed? 

MIL-M-38784  Tech  Manuals 

MIL-M-15071G 

OPNAVINST 
4160.2 
SECNAVINST 
5233. IB 


NOSC  GUIDES  Progress 

Reports 


Performance  spec 
Design  spec 

Subprogram  design  specs 
Common  data  base  design 
documentation 
Program  package 
Operators  manual 
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V .  PROCUREMENT 


NOSC  SUPPLY 

FAR/DFARS 

7.105 


Contract 

Service 


Contractor 


NOSC  SUPPLY 


SECNAVINST 
7000. 17B 
NAVMATINST 
7000. 17E 


NAVMATINST 
4000. 15A 


ONM  4335.8 


Contractor  Cost 
Performance 
Measurement  for 
Selected 
Acquisitions 

Data 


Performance 


1.  Have  contractual  services 
been  requested  from  Code  21? 

2 .  Has  complete  procurement 
package  data  been  compiled? 

3 .  Has  interface  between  the 
project  and  NOSC  procurement 
been  established? 

4.  Are  contractual  actions  con¬ 
sistent  with  NOSC  procurement 
practices  and  lead  time? 

5.  Is  an  Acquisition  Plan 
required? 

1.  Does  this  project  require 
contractor  assistance? 

2 .  Has  procurement  package  been 
planned,  prepared,  processed 
and  awarded? 

3 .  Does  the  contract  include 
sufficient  reporting  require¬ 
ments  to  provide  government 
visibility? 

1.  Will  the  provisions  of  this 
directive  to  Selected  Acqui¬ 
sitions  apply  to  this 
project? 


Are  the  contract  data 
requirements  specified? 

Are  all  specs  clear,  unambig¬ 
uous  and  unrestrictive? 

Are  contractor  performance 
evaluations  made? 


E.  SYSTEM  ENGINEERING 


MIL-STD-490 


Specifications 


Are  detailed  specs  prepared 
in-house  prior  to  procurement 
action? 

Are  design  specs  complete 
before  development? 

Does  the  Government  own  ail 
manufacturers'  drawings  and 
specs  developed? 

Are  design  specs  accurate  and 
adequate  so  that  competition 
will  be  possible  for  future 
procurements? 


paid'  4  of  10 


5.  Have  feasibility  and  effec¬ 
tiveness  studies  been  com¬ 
pleted  and  evaluated  before 
major  software  or  hardware 
commitments  are  made? 


NAVMATINST 

4120. 97B 

DOD  4120. 3M 

MIL-STD-680 

Standardization 

1. 

2  . 

Has  the  system  been  analyzed 
with  the  intent  of  standardi¬ 
zing  common  parts,  components, 
and  subsystems? 

Will  contract  specifications 
call  for  standardization? 

Review 

1. 

2. 

Is  a  preliminary  design 
review  schedule? 

Are  critical  design  reviews 
scheduled? 

MIL-STD-499A 

Systems 

Engineering 

1. 

Will  systems  engineering 
management  be  implemented? 

Management 

2  . 

Has  a  System  Engineering 
Management  Plan  (SEMP)  been 
prepared? 

MIL-STD-499A 

Integration/ 

Interface 

1. 

Has  the  impact  of  inter¬ 
dependent  tasks  on  ultimate 

project  completion  been 
evaluated? 


2.  Have  existing  systems  been 
studied  for  interface  or 
transition  to  the  new  system? 

3.  Have  detailed  interface 
characteristics  of  all  asso¬ 
ciated  systems  been  described 

4.  Has  a  person  been  assigned 
responsibility  for  interface 
control? 


DATA  MANAGEMENT 


NAVMATINST 
4000. 15A 


Data  Management  1 . 


DOD  5010.12 


K* 

V 


b 


2  . 

3  . 

4  . 


Will  project  technical  data 
management  be  in  accordance 
with  the  instructions? 

Has  a  data  manager  been 
appointed? 

Has  a  data  management  plan 
been  prepared? 

Has  a  data  depository  been 
established? 
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5.  Has  a  spec  tree  been 
established? 

MIL-STD-490  Specs 

MIL-M-1507 1G  Manuals 

MIL-M-38784  Manuals 

WS-8506  -  Software 

MIL-E-16400F  Electronic 

Equipment 

MIL-STD-1369  ILS 

MIL-STD-1472  Human 

Engineering 

MIL-STD-785  Reliability 

6.  Data  change  provisions 
established? 

7.  Have  Forms  DD-1423  been 
prepared 


INTEGRATED  LOGISTIC  SUPPORT  (ILS) 

1.  Has  a  training  plan  and 
schedule  been  prepared? 

2.  Does  the  training  plan  cover 
operators,  maintenance,  per¬ 
sonnel  and  support  personnel? 

3 .  Are  schedules  for  personnel 
training  consistent  with 
development,  production,  and 
installation  schedules? 

4.  Does  the  training  plan 
include  requirements  for 
training  devices  and  aids  to 
support  the  training  program? 

Validation  and  1.  Have  V&V  plans  been  prepared? 

Verification  2.  Have  site  activation  surveys 

( V&V)  and  reports  been  made? 

Maintenance  1.  Has  a  maintenance  plan  been 

developed? 

2.  Does  the  maintenance  plan 
identify  maintenance  function 
and  describe  maintenance 
levels  at  which  all  sub¬ 
system  and  components  are  to 
be  processed? 

3.  Will  standardization  goals 
be  established  early  in  the 
project  cycle? 

4.  Is  there  a  system  for 
reporting  malfunctions, 
problems,  etc? 


NAVMATINST  Training 

4000. 20B 

DOD  4100.35 

SECNAVINST 
4000. 20A 

MIL-STD-13  69  (EC) 
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Spares 


1.  Is  there  a  plan  for  develop¬ 
ing  spares  and  repair  parts 
requirements? 


NAVMATINST 
4855. 1A 


MIL-STD-480, 
481,  482  & 
483 


NAVMATINST 
4130. 1A 
OPNAVINST 

4130.1 

OPNAVINST 

4130.2 

DODDIR 

5010.19(D) 


Quality 
Assurance  (QA) 


1.  Has  QA  been  considered  for 
both  hardware  and  software? 

2.  Who  is  going  to  perform  QA  on 
tech  data  (manuals,  pro¬ 
cedures,  drawings)? 


Configuration 
Management  (CM) 


Software  (CM) 


1.  Is  CM  required  (hardware/ 
software) ? 

2.  Has  a  CM  plan  been  prepared? 

3.  Have  baselines  been 
established? 

4.  Have  all  items  been  identi¬ 
fied  and  listed? 

5.  Has  a  change  control  board 
(CCB)  been  established? 


6.  Are  all  changes  reported  to 
the  CCB? 

7.  Does  the  CCB  secretary  report 
results  of  each  meeting  and 
maintain  records? 

8.  Do  all  subsystems  have  repre¬ 
sentatives  and  alternates? 


NAVMATINST  Human  Factors 

3900. 9A 


MIL- STD-14 72 A 
MIL-H-4  68  55 

NAVMATINST  Value 

4858. 8A  Engineering 

(VE) 


FAR,  SECT  1, 
PART  17 


MIL-V-3  8  3  52 


MIL-STD-100A 

MIL-D-1000 

MIL-D-1000/2 

MIL-D-1000/1 

MIL-D-5480 


1.  Have  human  engineering  con¬ 
siderations  been  imposed  upon 
concept  formulation,  design 
and  development  of  the  system' 

2.  Has  a  human  engineering  plan 
been  prepared? 

1.  Has  the  program  and  budgetary 
planning  provided  for  VE  con¬ 
siderations  and  effort  during 
the  contract  definition  and 
engineering  development 
phases? 

2 .  Has  a  VE  program  requirement 
spec  and  plan  been  prepared 
for  application  in  the  system 
development  contract? 

3.  Do  project  management  plans 
provide  for  the  continuing 
integration  of  VE  considera¬ 
tions  throughout  the  life 
cycle  of  the  systems? 

1.  Will  drawings  be  prepared  in 
accordance  with  MIL-STDS? 

2.  Is  there  a  schedule  for  the 
acquisition  and  development 
of  provisioning  documentation 


Drawings 


Support 

Equipment 


Packaging  and 
transportation 


3 .  Does  the  schedule  for  pro¬ 
curement  and  delivery  of 
parts  conicide  with  system 
delivery? 

1.  Is  there  a  plan  for  the  deve¬ 
lopment  of  support  equipment 
requirements? 

2 .  Does  the  schedule  for  pro¬ 
curement  and  delivery  of 
approved  support  equipment 
coincide  with  system  delivery? 

1.  Have  packaging  and  transpor¬ 
tation  requirements  been 
established  for: 


Operational  items 

End  items  to  be  repaired 

Spare  and  repair  parts 


i 


IX.  PRODUCT  ASSURANCE 


NAVMATINST 

3000.1 

Reliability 

1. 

Has  a  reliability/maintain¬ 
ability  program  been 
established? 

NAVSEA INST 

3900 . 2 
NAVELEXINST 

4858.2 

Reliability  and 
Maintainability 
Reliability 

2  . 

Have  the  SYSCOM's  particular 
requirements  been  satisfied? 

NAVELEXINST 

4858.3 

NAVAIRINST 

13070.2 

Maintainability 

Reliability  and 
Maintainability 

3  . 

Have  the  Center's  require¬ 
ments  been  satisfied? 

NAVMATINST 

4855.1 

Quality 

Assurance  (QA) 

1. 

Has  a  Quality  Assurance 
program  been  established? 

NAVSEAINST 

4855.5 

Quality 

Assurance  (QA) 

2  . 

Have  the  SYSCOM's  particular 
requirements  been  satisfied? 

NAVELEXINST 

4855.2 

NAVAIRINST 

5400.23 

Quality 

Assurance  (QA) 

Qual ity 

Assurance  (QA) 

3  . 

Have  the  Center's  require¬ 
ments  been  satisfied? 

NAVSEA  Weapons  and 

OD  46574B  Combat  Systems 

Product  Assurance 
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U  O 


NOSC  TD  870 

Product 

Assurance 

Requirements 

1.  Has  a  product  assurance 
program  been  established? 

NOSC  TD  432 

Product 

2 .  Has  the  product  assurance 

MIL-STD-470/785 

882/1472 

NAVMAT  P-949 

Assurance 

Guidelines 

program  plan  been  completed 
and  approved? 

\V.*. 

TEST  AND  EVALUATION  (T&E) 


ONRINST 
5210. 2B 
OPNAVINST 
3690. 10B 
SECNAVINST 
5233. IB 
NAVMATINST 
3960. 6A 
NAVMATINST 
3960. 7A 

associated  subsystems? 

6.  Have  training  programs  been 
planned  and  scheduled  for 
test,  evaluation,  installation 
and  maintenance  personnel? 

7.  Does  the  test  plan  clearly 
delineate  the  responsibilities 
and  relationships  of  all 
agencies  (contracting  and 
government)  in  preparing  for 
the  test  and  evaluation  phase? 

8 .  Have  test  procedures  been 
prepared? 

9.  Has  liaison  been  established 
with  the  Chief  and  the 
Commander,  Operational  Test 
and  Evaluation  Force 
(COMOPTEVFOR)  early  in  the 
project  cycle? 

10.  At  completion  of  test  and 
evaluation,  will  all  end 
items  be  delivered  con¬ 
currently  with  documentation 
and  evaluation  reports  to 
the  user? 


Test  and 
Evaluation 
(Software/ 
Hardware) 


1.  Have  overall  levels  of 
acceptable  performance  been 
established  by  users? 

2 .  Has  a  T&E  Plan  been  developed? 

3.  Does  the  T&E  Plan  agree  with 
development  schedule? 

4 .  Does  component  testing  take 
place  prior  to  system  testing? 

5.  Does  the  test  plan  include 
testing  of  interfaces  with 
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XI.  SECURITY 


XII. 


U 


OPNAVINST 
5510. 1G 

DOD  5200. 1R 


1.  Has  an  appropriate  Security 
Classification  Plan  or  Guide 
been  developed  for  the  system, 
components,  and  documents? 


SAFETY 


OPNAVINST  System  Safety 

5100. 8F  Program  Plan 

(SSPP) 

MIL-STD-882A 

NAVMATINST 
5100. 6A 


1.  Has  a  SSPP  been  prepared  in 
accordance  with  MIL-STD-882A? 

2.  Have  the  SSPP  requirements 
been  included  in  the  funding 
schedules,  milestones,  and 
WBS? 


NOSCINST 
5100.8 
BUMEDINST 
6470. 1A 
NAVSEA 
OP3565 
NAVAIR 
6-1-529 
SPAWAR 

0967-LP-624-6010 
NOSCINST 
2470. 1C 
OPNAVINST 
2410. 11B 


Electromagnetic 

Environment 


1.  Does  the  electromagnetic 
environment  provide  for  pro¬ 
tection  against  microwave 
hazards? 

2.  Have  Center  frequency  coordi¬ 
nation  and  supporting  pro¬ 
cedures  been  followed? 
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Appendix  7J 


Project  Progress  Report 
(See  subsection  7.11) 


24/ 


PROJECT  PROGRESS  REPORT 


PAGE  2 


ANTICIPATED  CRITICAL  PROBLEMS  /State  Nature  ut  problem  impact  on  protect,  action  to  be  taken  or  recommended,  and  when  for  each  problem) 


Instruction  for  preparation  and  Completion  of 
Project  Progress  Report 


Block 

Entry  Item 

Instruction 

1 

From 

Enter  originator's  name  and  code. 

2 

To 

Enter  addressee's  name  and  code. 

3 

NOSC  Project 

Enter  the  four  digit  project  number. 

4 

Project 

Enter  the  short  title  of  the  program 
or  project.  Identifying  acronym  may 
be  used,  if  applicable. 

5 

Reporting  Period 

Enter  the  ending  date  of  the  period 
covered  in  the  progress  report. 

6 

Date 

Enter  the  preparation  date  of  the 
report. 

7 

Sub-Project 

Enter  the  System  Command's  identi¬ 
fication  number. 

8 

Task 

Enter  the  System  Command's  task 
number . 

9 

Sponsor 

Enter  the  System  Command  that 
sponsors  the  project. 

10 

Principal  Investigator 

Enter  the  name,  code  and  telephone 
number  of  the  person  responsible 
for  directing  the  project  effort. 

11 

Associate  Investigator 

Enter  the  name,  code  and  telephone 
number  of  the  primary  person 
responsible  for  the  technical  work. 

12 

Due  this  Reporting 
Period 

Provide  names  and  status  of  acti¬ 
vities  scheduled  to  be  completed 
during  this  reporting  period. 

13 

Due  Next  Reporting 
Period 

Provide  names  and  status  of  acti¬ 
vities  scheduled  to  be  completed 
during  the  next  reporting  period. 

14 

Activities  in  Process 

Provide  names  and  status  of  acti¬ 
vities  in  process  that  are  related 
to  the  accomplishment  of  milestones. 

a'-' 
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V  % 


Block 


15 

16 
17 


18 


19 


20 


21 


22 


23 


24 


Entry  Item 


Instruction 


Milestone  Status  Number  Enter  sequential  number  identified. 
Milestone  Identification  Enter  name  of  milestone. 


Milestone  Date 


Check  One  - 
Made/Missed 


Enter  the  milestone  schedule  date. 
If  the  milestone  schedule  date  is 
revised,  enter  the  revised  date. 
Enter  date  that  milestone  is 
completed . 

Enter  check  mark  under  appropriate 
colunm  as  to  milestone  made  or 
missed. 


Totals 


Discuss  Milestones 
Missed 


Current  Critical 
Problem 


Enter  number  milestones  made  and 
number  milestones  missed  in  appro¬ 
priate  colunm. 

Provide  reasons,  effect  on  program/ 
project,  remedial  action  taken  or 
to  be  taken,  and  when. 

Provide  the  nature  of  critical 
problem,  impact  on  project,  action 
to  be  taken  or  recommended,  and  when. 


Anticipated  Critical 
Problem 


Provide  nature  of  anticipated 
problem  and  remediall  action  planned 
or  taken. 


Contracts/ 
Purchase  Orders 


Provide  contractor's  name,  if  appli¬ 
cable,  item  or  name  of  procurement, 
due  date,  and  status. 


Security 

Classification 


Enter  the  security  classification, 
i.e.,  CONFIDENTIAL,  SECRET,  etc., 
in  the  upper  left-hand  and  lower 
right-hand  corners  of  each  page. 


Appendix  7K 


Acquisition  Plan  Outline 
(See  subsection  7.12.2) 


7-90 


Acquisition  Plan  Outline 


Para . 


1.0 
2 . 0 

2.1 

2.2 

2 . 3 

2 . 4 

2 . 5 

2.6 

2.7 

2.8 

2.9 

2.10 
2.11 

2 . 12 

3.0 

3.1 

3.2 
3  .  3 
3 . 4 


Title 

LIST  OF  TABLES 

Description  of  Program/System  Commodity,  etc. 

Section  A  -  Acquisition  Background  and  Objective 
-  (FAR  7.105(a) ) 

Statement  of  Need  (FAR/NARSUP  7.105(a)(1)) 

Applicable  Conditions  (FAR  7.105(a)(2)) 

Cost  (FAR  7 . 105(a) (3) ) 

Capability  of  Performance  (FAR  7.104(a)(4)) 

Delivery  or  Performance  Period  Requirement 
(FAR  7.105(a)  (5)  ) 

Trade-offs  (FAR  7.105(a)(6)) 

Risks  (FAR/DFARS/NARSUP  (7.105(a)(7)) 

Applicability  of  DCP,  DSARC  and/or  Internal  Service  Review 
( DFARS/NARSUP  7 . 105 ( a )  ( 10 ) ) 

Approval  for  Operational  Use  ( DFARS  7 . 105  (a)  (71) ) 

Miestone  Chart  (DFARS/NARSUP  7 . 105 (a) (721) ) 

Milestones  for  Updating  the  Acquisition  Plan 
(DFARS  7.105(a)  (73)  ) 

Background  and  Acquisition  History 

Section  B  -  Plan  of  Action  (FAR  7.105(b)) 

Sources  (FAR  7 . 105 (b) (1) ) 

Competition 

Source  Selection  Procedures 
Contracting  Considerations/Contract  Type 


Para. 

3.5 

3 . 6 

3.7 

3.8 

3.9 

3 . 10 

3.11 

3 . 12 

3 . 13 

3 . 14 

3 . 15 

3 . 16 

3 . 17 

3 . 18 

3.19 


Title 

Budgeting  and  Funding  (FAR/NARSUP  7.105(b)(5)) 

Product  Description  (FAR  7.105(b)(6)) 

Priorities,  Allocations  and  Allotments  (FAR  7.105(b)(7)) 

Contractor  vs  Government  Performance  (FAR  7.105(b)(8)) 

Management  Information  Requirements 
(FAR/NARSUP  7.105(b)(9)) 

Make  a  Buy  (FAR  7.105(b)) 


Test  and  Evaluation  (FAR  7 . 105 (b) ( 11) ) 

Logistics  Considerations:  Support,  R&M,  QA,  data, 
standardization.  (FAR/DFAR/NARSUP  7 . 105 (b) ( 12 ) ) 

Government  Furnished  Property  (FAR  7 . 105 (b) ( 13 ) ) 

Government  Furnished  Information  (FAR  7 . 105  (b)  ( 14 ) ) 

Environmental  Considerations  (FAR  7 . 105 (b) ( 15 ) ) 

:  curity  Considerations  (FAR  7 . 105 (b) ( 16) ) 

Other  Considerations  (FAR/NARSUP  7 . 105 (b) ( 17 ) ) 

Milestones  for  the  Acquisition  Cycle 

Identification  of  Participants  in  AP  Preparation 

Acquisition  Approach  for  Each  Proposed  Contract  - 
Ref  subitems  "a"  through"m''  (DFARS  7 . 105  (b)  (70)  ) 


m* 


3.20 


ACQUISITION  PLAN  REQUIRED  TABLES 


Table  Title 

I  Projected  Funding  (Ref  para  2.3  Cost) 

II  Personnel  Requirements  (Ref  para  2  A 
Capability  of  Performance) 

III  Milestone  Chart  (Ref  para  2.10  Milestone 
Chart) 

IV  Contracts  in  Excess  of  $5d0M  (Ref  para 
2.12  Background  and  History) 

V  Source  Selection  List  (Ref  para  3.1 
Sources) 

VI  Budgeting  and  Funding  (Ref  para  3.5 
Budget  and  Funding) 


CONTENTS 


Introduction  . 

References . 

Outline . 

Summary . 

Definitions . 

Systems  Engineering  . . 

Action  Items . 

The  Pattern  of  Success 
The  Project  Manager . . 


SECTION  8 

SYSTEMS  ENGINEERING 
J.  Townsend,  Code  934 


8.1  INTRODUCTION 

8.1.1  References 

MIL-STD-881A  Work  Breakdown  Structures  for  Defense  Material  Items 
MIL-STD-499A  Systems  Engineering 
NOSC  TD  108  Project  Manager’s  Guide 

NOSC  TD  250  Suggestions  for  Designers  of  Navy  Electronic  Equipment 

8.1.2  Outline 

Introduction 
References 
Outline 
Summary 
Definitions 
Systems  Engineering 
Action  Items 
The  Pattern  of  Success 
The  Project  Manager 

8.1.3  Summary 

Systems  engineering  is  both  an  engineering  art  and  a  management  discipline.  This  module  for  project 
managers  is  a  concise  treatment,  which  only  touches  on  the  key  topics  and  issues,  of  a  very  complex 
topic.  Systems  engineering  is  defined  and  then  related  to  project  management.  The  tasks  of  systems 
engineering  and  systems  engineering  management  are  described,  and  commonly  encountered  project 
problems  are  highlighted.  Planning  and  management  tools  of  importance  to  project  managers  are  related 
from  a  systems  engineering  perspective.  Because  of  the  nature  of  systems  engineering,  this  module  tends 
to  tie  together  other  course  topics. 


8.2  DEFINITIONS 

Systems  Engineering:  Systems  engineering  is  the  application  of  scientific  and  engineering  efforts  to: 

a.  Transform  an  operational  need  into  a  description  of  system  performance  parameters  and  a 
system  configuration  through  the  use  of  an  iterative  process  of  definition,  synthesis,  analysis, 
design,  test,  and  evaluation. 

b.  Integrate  related  technical  parameters  and  ensure  compatibility  of  all  physical,  functional,  and 
program  interfaces  in  a  manner  that  optimizes  the  total  system  definition  and  design. 
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c.  Integrate  reliability,  maintainability,  safety,  survivability,  human,  and  other  such  factors  into 
the  total  engineering  efforts  to  meet  cost,  schedule,  and  technical  performance  objectives 
(MIL -STD-499). 

Chief  Systems  Engineer:  The  individual  tasked  to  supervise  the  systems  engineering  tasks. 
Organizationally,  the  chief  systems  engineer  is  also  a  deputy  project  manager  who  is  responsible  for 
monitoring  and  managing  the  technical  progress  of  the  project. 

8.3  SYSTEMS  ENGINEERING 

Systems  engineering  is  the  “glue”  that  binds  the  project  product  to  the  original  need.  The  original 
requirements  statement  is  provided  in  operational  terms  and  is  usually  too  incomplete  to  support  technical 
design  decisions.  Systems  engineering  tasks  analyze  and  refine  requirements,  translate  operational 
requirements  into  technical  requirements,  and  provide  the  technical  project  monitoring  and  controls 
that  ensure  that  the  product  meets  performance  requirements  within  the  cost  and  schedule  goals  of 
the  project.  Figure  8.1  through  8.5  reflect  some  of  the  issues  of  concern  to  those  involved  in  systems 
engineering. 

Often  the  requirements  statement  is  inadequate  to  support  the  project  effort.  Requirements  may 
be  excessive  to  any  imagined  mission,  attempt  to  violate  physical  laws,  or  be  incompletely  stated.  Any 
of  these  problems  must  be  resolved  before  resources  are  committed  to  the  design. 

Other  problems  are  created  by  the  need  to  “know  everything”  prior  to  beginning  the  project  when, 
in  fact,  the  project  may  be  advancing  the  state-cf-the-art.  Systems  engineering  disciplines,  combined 
with  the  information  resources  of  the  Center,  enable  the  project  team  to  identify  decision  points  and 
apply  appropriate  resources  to  gain  sufficiently  accurate  information  for  those  decisions. 

The  chief  (or  lead)  systems  engineer  is  (normally)  a  deputy  project  manager.  In  this  role,  he 
establishes  technical  process,  ensures  communication  within  the  technical  team,  and  oversees  the  internal 
review  processes.  Other  major  management  tasks  include  costing  disciplines  and  risk  management. 

Systems  engineering  includes  extensive  costing  disciplines.  These  range  from  techniques  of  more 
accurately  estimating  and  projecting  project  task  costs  to  techniques  of  establishing  and  tracking  design- 
to-cost  targets  and  life-cycle  cost  targets.  These  disciplines  are  very  important  to  the  successful 
employment  of  the  project  product;  decisions  made  in  the  conceptual  phase,  through  which  only  a 
few  percent  of  the  total  project  cost  is  actually  expended,  often  affect  total  life-cycle  costs  by  as  much 
as  two  orders  of  magnitude.  Systems  engineering  allows  these  cost  factors  to  be  properly  incorporated 
in  the  project  decisions  before  the  actual  dollars  can  be  known. 

Risk  management  is  a  tremendously  important  field,  but  one  which  is  frequently  overlooked.  The 
management  of  technical  risks  is  a  systems  engineering  function,  but  many  of  the  risk  management 
techniques  bear  heavily  on  project  planning.  The  management  of  project  risks  (cost,  schedule,  and 
political  risks)  is  a  project  management  responsibility;  nevertheless,  the  project  manager  may  depend 
heavily  on  the  systems  engineer  for  expertise  in  project  risk  management  as  well.  The  techniques  of 
risk  management  are  now  expressed  in  very  well  defined  procedures.  Projects  employing  risk  management 
are  not,  of  course,  guaranteed  success,  but  project  managers  who  manage  risks  well  are  very  significantly 
more  successful  than  those  who  do  not. 

Throughout  the  project  life,  the  systems  engineering  depends  heavily  on  project  team  experts  in 
design,  product  assurance,  test  and  evaluation,  and  other  project  disciplines.  The  most  critical  “original” 
systems  engineering  tasks  occur  during  the  requirements  definition  and  conceptual  phases  where  solid 
engineering  decisions  must  be  made  before  much  of  the  information  used  by  the  other  disciplines  is 
developed.  It  is  during  these  phases  that  operational  requirements  are  transformed  into  technical 
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Figure  8.1.  Level  of  information  versus  confidence  and  effort  to  obtain. 
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Figure  8.2.  Project  size  versus  lange  of  project  issues. 
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requirements.  In  the  validation,  design,  production,  and  deployment  phases,  the  systems  engineering 
role  becomes  increasingly  a  design  assurance  role,  coordinating  the  other  technical  tasks.  As  the  project 
product  reaches  deployment,  virtually  all  tasks  are  being  executed  by  the  designated  team  experts,  and 
the  project  manager  often  assumes  the  role  of  the  systems  engineer.  Exceptions  to  this  normal  trend 
include  very  large  projects  and  projects  with  continuing  product  improvement  efforts. 

Systems  engineering  interacts  with  design  to  determine  the  appropriate  levels  of  standardization 
and  design  ownership;  to  ensure  EMX  issues  (these  are  issues  concerned  with  an  integrated  approach 
to  the  study  of  the  total  effect  of  electromagnetics)  are  properly  accounted  for;  and  to  make 
build/buy/modify  decisions.  Systems  engineering  and  product  assurance  cooperated  to  determine  level 
of  repair  and  govemment/industry  design  interfaces.  These  decisions  apply  to  both  hardware  and  software 
design  issues.  Another  important  systems  engineering  design  task  is  the  control  of  hardware/software 
interfaces.  Each  of  these  areas  create  test  and  evaluation  issues  which  the  test  director  and  the  systems 
engineer  resolve  together. 


8,4  ACTION  ITEMS 

a.  Select  and  involve  a  chief  systems  engineer  in  the  proposal  stage  and  carry  his  involvement 
throughout  the  project.  Changes  in  systems  engineer  have  had  a  more  negative  impact  on  the 
project  historically  than  changes  in  the  project  manager.  The  chief  systems  engineer  should 
be  technically  knowledgeable  in  the  key  project  issues,  able  to  communicate  and  work  well 
with  others,  dedicated,  practical,  and  inquisitive. 

b.  Make  use  of  the  extensive  support  assets  in  Codes  16,  17,  91,  92,  93,  95,  and  96.  These  codes 
have  people  experienced  in  the  many  critical  disciplines  associated  with  systems  engineering. 

c.  Make  use  of  NOSC  TD  108.  The  tips  and  information  contained  in  NOSC  TD  108  are  project- 
tested  and  proven  practical. 

d.  Formulate  the  project  work  breakdown  structure  in  accordance  with  M1L-STD-881,  but  extend 
the  WBS  to  the  least  significant  configuration  item.  In  order  to  accommodate  the  natural  desire 
to  organize  the  project  functionally  (which  is  more  convenient  in  establishing  work  agreements 
and  intercode  funding),  use  the  last  digit  of  each  work  package  to  encode  the  performing 
organization.  When  using  computer  support,  the  program  can  track  the  project  using  the  WBS 
configuration  or  using  the  project  functional  organization. 


8.5  THE  PATTERN  OF  SUCCESS* 

The  role  of  the  project  manager  is  to  acquire  equipment  which  will  perform  the  required  functions 
at  an  affordable  price  and  by  the  time  they  are  needed.  In  these  days  of  constrained  budgets,  “affordable” 
may  be  defined  as  the  least  total  cost  to  the  government  only  with  the  proviso  that  the  functions  to 
be  performed  are  worth  that  expenditure.  Literally  hundreds  of  studies  have  looked  at  defense  acquisitions 
over  the  past  30  years.  Reading  the  conclusions  and  recommendations  from  a  1949  report  is  like  reading 
the  results  of  a  1974  report.  Project  after  project  fails  to  achieve  its  goals.  Each  report  is  clear;  projects 
fail  to  meet  performance  objectives,  overrun  costs  by  150  percent,  and  slip  schedules  25  to  50  percent. 
In  the  vast  majority  of  cases,  the  project  goals  were  not  achieved  for  the  following  reasons;  misspeci- 
fication  (usually  gross  overspecification  creating  artificial  technical  problems),  failure  to  manage  risks, 
obscuring  of  the  project  goals  through  extraneous  paperwork  requirements,  and  failure  to  define 
adequately  what  is  required.  These  reasons  are,  however,  only  the  symptoms  of  underlying  problems 
‘Excerpted  and  adapted  from  NOSC  TD  108. 


in  the  acquisition  community.  The  TELCAM  project  looked  at  successes  and  failures  in  industry  as 
well  as  government;  the  successful  project  has  the  same  traits  whether  in  industry  or  in  government. 
An  acquisition  project  also  shares  many  traits  with  a  small  business,  so  TELCAM  also  solicited 
information  from  the  Small  Business  Administration.  Again,  success  is  a  pattern,  whereas  failure  is 
a  deviation  from  that  pattern.  The  major  difference  between  failures  in  industry  and  failures  in 
government  is  that  a  failing  project  in  industry  is  usually  rapidly  terminated;  the  government  failure 
usually  plods  on  to  an  elegant  wreck. 

What  is  the  elusive  pattern  of  success?  The  projects  cited  as  successes  will  have  two  main  features: 

A  strong,  knowledgeable  project  manager  who  acts  as  the  ultimate  authority  for  all  project 
matters — tasking,  budgeting,  technical  decisions,  etc. 

A  small,  dedicated  team  executing  project  tasks. 

The  key  words  above  are  strong,  knowledgeable,  ultimate  authority,  small,  dedicated,  and  team. 
Excellent  studies  of  the  nuclear  power  program,1  the  Polaris  system,2  and  NTDS3  are  available  which 
show  these  forces  at  work.  “Strong”  appears  to  be  a  peculiar  necessity  in  the  government  projects, 
as  each  success  seems  to  attain  that  status  in  spite  of  “the  system.”  An  ultimate  goal  of  acquisition 
R&D  must  be  to  change  “the  system”  to  allow  average  individuals  to  be  successful  project  managers. 
Until  that  goal  is  reached,  there  is  still  enough  of  a  task  to  create  knowledgeable  managers.  Some 
spectacular  failures  have  been  managed  by  strong,  unknowledgeable  individuals.  A  project  needs  a 
strong  champion  in  order  to  “steal”  sufficient  authority  to  become  a  purposeful  autonomous  entity, 
but  authority  unwisely  wielded  is  disaster.  The  government  project  manager  is  not  held  accountable 
for  his  actions;  accountability  is  the  “quality  assurance”  incentive  used  to  check  authority  in  industry. 

The  first  procurement  of  muskets  by  the  Army  in  1798  seemed  straightforward.  Eli  Whitney 
promised  to  produce  and  deliver  muskets  built  by  assembly  line  techniques,  and  using  interchangeable 
parts,  within  8  months;  actual  delivery  was  made  10  years  late.  Our  military  procurement  problems 
actually  predate  the  nation  itself;  one  verse  of  “Yankee  Doodle”  went 

“And  there  we  saw  a  thousand  men 

As  rich  as  Squire  David, 

And  what  they  wasted  every  day 

I  wish  it  could  be  saved” 

referring  to  one  of  General  Washington’s  encampments. 

On  March  27,  1794,  Congress  appropriated  5768,000  for  the  construction  and  manning  of  six  frigates. 
Each  frigate  was  to  cost  $100,000.  When  the  UNITED  STATES,  CONSTITUTION,  and  CONSTEL¬ 
LATION  were  finally  launched  in  1797,  each  cost  close  to  $300,000. 

The  innumerable  instances  of  procurement  problems  which  have  occurred  repeatedly  over  the  years 
have  only  worsened  with  time.  The  evolving  acquisition  system  operated  in  ignorance  in  the  1790s, 
and  this  basic  ignorance  exists  today  throughout  the  acquisition  community.  Less  than  30  percent  of 
the  project  managers  interviewed  by  TELCAM  knew  the  operational  objective  of  their  project;  only 
5  percent  could  relate  technical  features  of  the  project  to  operational  considerations.  Only  2  percent 
of  the  project  managers  were  aware  of  any  actions  being  taken  on  their  project  to  reduce  risks  of  any 
kind;  only  half  knew  what  progress  was  being  made  or  what  difficulties  were  being  encountered  on 
major  tasks!  Under  these  circumstances,  it  is  a  wonder  even  greater  failures  do  not  occur  than  those 
already  found.  One  of  the  major  factors  aggravating  this  situation  is  the  lack  of  project  manager 

'Nuclear  Navy  (1946-1962),  RG  Hewlett  and  F  Duncan,  Chicago,  1974 

2The  Polaris  System  Development,  Sapolsky,  Harvard,  1972 

3The  NTDS  Development,  RW  Graf,  United  Research,  Inc,  29  Jan  1964 
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accountability  for  the  end  item  in  the  field.  A  project  manager  may  only  influence  4  years  of  project 
life,  whereas  the  end  item  may  be  in  use  for  over  20  years.  The  project  manager  determinations  affect 
all  but  a  few  percent  of  the  total  life  cost  of  a  project. 

Figure  8.6  shows  the  percentage  of  total  ownership  costs  committed  during  conceptual  planning, 
design,  development,  acquisition,  and  operations  for  past  major  programs.  In  the  past,  decisions  made 
during  the  concept  and  planning  phases  committed  70  percent  of  the  total  life-cycle  cost  funds  of  a 
program,  while  design,  development,  acquisition,  and  operations  accounted  for  only  30  percent  of  that 
total  cost.  The  effects  of  application  of  life-cycle  cost  analysis  through  the  planning  and  RDT&E  phases 
of  a  program,  and  the  “design  to  cost”  concept  on  new  programs  are  expected  to  change  this  distribution 
considerably  by  affecting  a  larger  portion  of  that  early  70  percent  commitment. 

Notice  that  90  percent  of  the  total  project  cost  is  fixed  after  only  10  percent  of  the  funds  have 
been  expended.  Unless  the  project  manager  assumes  responsibility  for  the  total,  it  is  impossible  to  attain 
the  lowest  possible  project  cost.  Considering  that  support  costs  may  be  altered  by  as  much  as  two  orders 
of  magnitude  by  decisions  in  the  conceptual  phase,  it  can  be  seen  that  a  naive  approach  to  project 
management  can  have  a  devastating  effect  on  operating  budgets;  likewise,  sound  management  can  lead 
to  a  highly  efficient  use  of  funds. 

The  acquisition  manager  should  try  to  obtain  equipment  which  is  fully  capable  of  doing  the  required 
job  and  which  has  the  following  characteristics: 

Reliable 

Maintainable 

Supportable 

Procurable 

Producible. 


8.6  THE  PROJECT  MANAGER 

A  manager  and  a  project  are  established  for  a  single  purpose.  It  makes  no  difference  that  the  project 
is  designated  a  major  program  and  its  manager  a  program  manager  or  that  the  project  is  called  a  tasking 
with  a  project  engineer  in  charge.  Program  manager,  acquisition  manager,  and  project  manager  are, 
for  the  purposes  of  the  discussion,  synonymous,  being  different  in  scope  but  like  in  character.  Project 
management  is  the  planning,  executing,  directing,  and  controlling  of  a  relatively  short-term  project 
or  systems-oriented  organization  established  for  the  completion  of  specific  goals.  Those  specific  goals 
will  be  the  acquisition  and  implementation  of  military  equipment  and  the  subgoals  associated  with  each 
phase  of  the  cradle-to-grave  life  of  that  equipment;  however,  the  principles  presented  are  basic,  universal, 
and  adaptable  to  many  other  project  circumstances. 

The  project  manager  ideally  will  plan,  organize,  monitor,  and  direct  the  project  to  its  goals  as 
effectively  as  possible.  Efficiency  is  a  secondary  consideration,  since  maximum  efficiency  often 
compromises  effectiveness.  It  is  generally  agreed  that,  in  the  competitive  atmosphere  of  military  affairs, 
ineffectiveness  is  catastrophic.  Organizations  which  manage  for  efficiency  are  called  functional 
organizations.  In  executing  his  tasks,  the  project  manager  will  draw  on  expert  assistance  from  many 
functional  areas  and  will  establish  lines  of  organization  control  which  will  allow  him  to  manage  efficiently. 
In  general,  he  has  two  cardinal  rules  to  follow: 

Do  not  do  it  yourself — accomplish  through  the  project  oiganization. 

Organize  your  resources  to  fit  the  project — be  prompt  and  precise  in  defining  the  organization. 
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Figure  8.6.  Systems  funds  committed  by  initial  planning  decisions. 


The  project  organziation  exists  within  an  organization  (which  will  normally  be  functionally 
organized).  In  order  to  reach  its  goals,  it  must  live  within  the  chain  of  command  of  its  parent  organization, 
and  it  must  establish  a  chain  of  command  within  itself.  A  chain  of  command  is  an  organization  of 
three  elements:  responsibility,  authority,  and  accountability.  Usually  a  project’s  charter  will  define  the 
project  goals  without  mention  of  these  three  elements.  Organization  instructions  may  define  project 
manager  responsibilities  in  a  general  way  with  only  implications  of  assigning  authority  and  no  actual 
accountability.  In  practice,  the  project  manager  should  assume  that  he  is  fully  responsible  for  meeting 
his  project  goals  and  he  should  assume  all  the  authority  that  he  is  allowed  by  law  and  by  his  supervision 
to  meet  his  reponsibilities.  Within  the  project  organization,  he  will  clearly  assign  responsibilities,  delegate 
appropriate  authority,  and  hold  accountable  each  responsible  individual.  The  key  to  his  success  will 
often  be  his  authority  and  his  ability  to  exercise  and  delegate  it.  Outside  the  project,  the  manager  should 
elicit  the  cooperation  of  those  who  have  authority  above  him,  to  ensure  that  he  is  backed  up  by  keeping 
his  chain  of  command  informed  truthfully,  concisely,  and  specifically.  Authority  is  the  power  to  make 
decisions.  It  is  important  to  remember  that  small  decisions  must  be  made.  A“no  decision” is  worse  than 
a  “wrong  decision  ’’  because  with  the  wrong  decision  the  manager  knows  what  he  did  and  can  correct 
it;  with  no  decision,  the  situation  will  inevitably  grow  worse,  perhaps  without  any  indication  of  the 
appropriate  corrective  action.  Admiral  Nimitz  was  reputed  to  have  said,  “the  time  for  taking  all  means 
for  a  ship’s  safety  is  while  you  are  still  able  to  do  so.”  Decisions  are  required  to  solve  problems;  solutions 
usually  result  from  perspiration — not  inspiration.  When  a  manager  has  a  problem,  he  has  basically 
two  methods  available  to  solve  it;  the  important  thing  is  that  the  decision  be  made. 


PROBLEM  SOLVING 


Classical  Method 

1.  What  is  the  problem? 

2.  What  are  the  alternatives? 

3.  What  is  the  best  alternative? 


Scientific  Method 

1.  Define  the  problem 

2.  State  objectives 

3.  Formulate  a  hypothesis 


4.  Collect  data 

5.  Classify,  analyze,  and  interpret  data 
against  the  hypothesis 

6.  Draw  conclusions,  generalize,  restate,  or 
develop  new  hypotheses. 

The  solution  should  be  kept  in  perspective  by  asking,  “Is  it  adequate?”  and  “Is  it  too  elaborate?” 

In  order  to  make  decisions,  the  manager  must  be  informed.  The  manager  uses  the  project 
organization  and  procedures  to  keep  informed  of  project  activities,  usually  using  some  form  of  convenient 
management  information  system.  Again,  the  solution  is  tailored  to  the  needs.  On  small  projects,  the 
project  manager  will  keep  directly  informed  about  all  the  specifics  of  his  project.  On  large  projects, 
the  project  manager  will  rely  mainly  on  reports  and  plans  and  will  focus  on  exceptions  to  the  overall  plan. 

In  order  to  obtain  up-the-line  cooperation  to  get  higher  order  decisions,  the  project  manager  should 
know  his  own  project  and  also  related  projects.  In  knowing  his  own  project,  the  manager  can  confidently 
relate  accurate  information  to  his  superiors.  This  confidence  and  frankness  can  play  a  role  in  generating 
trust  which  will  be  valuable  if  problems  requiring  outside  assistance  arise.  Also,  the  knowledge  of  other 
projects  will  assist  the  manager  in  recognizing  the  parent  organization’s  perspective  and  in  establishing 
a  priority  to  obtain  the  needed  decision.  Avoiding  “tunnel  mindedness”  can  be  very  helpful  when 
competing  for  a  share  of  limited  organization  resources — especially  funding.  Avoid  “buttering  up” 
reports  to  show  only  good  news;  major  problems  cannot  be  covered  up  and  will  torpedo  this  facade. 
The  project  must  satisfy  the  parent  organization’s  goals. 

The  Project  Manager’s  Guide  (NOSC  TD  108)  is  offered  in  recognition  of  the  fact  that  most  project 
managers  are  good  technical  men  who  may  be  inexperienced  managers.  It  is  also  an  attempt  to  offer 
practical  methods  to  implement  the  recommendations  of  the  various  government  studies  on  reducing 
costs  (see  appendix  B  of  the  guide),  many  of  which  are  not  addressed  by  directives  and  instructions. 
It  is  hoped  that  the  guide  will  serve  as  a  useful  navigational  tool  for  the  manager  as  he  weaves  his 
project’s  course  through  instructions,  budgets,  specifications,  and  the  like  to  a  successful  implementation 
in  the  Fleet. 
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SECTION  9 

MAJOR  SYSTEMS  ACQUISITION 
Dr.  Julius  Hein* 


9.1  INTRODUCTION 


9.1.1  References 

“Project  Management:  An  Overview,’’  James  J.  O’Brien,  Project  Management  Quarterly,  Volume 
VIII,  Number  3,  September  1977.  SCAN. 
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Uniformed  Services  University  of  the  Health  Sciences 
•Defense  Systems  Management  College,  St.  Louis,  Missouri. 
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Organizations  and  Functions — Office  of  the  Secretary  of  Defense 
Immediate  Offices  of  Secretary  and  Deputy 
Secretary  of  Defense 
Under  Secretary  of  Defense  (Policy) 

Under  Secretary  of  Defense  (Research  and  Engineering) 

Assistant  Secretary  of  Defense  (Controller) 

Assistant  Secretary  of  Defense  (Manpower,  Installations,  and  Logistics) 
Director  of  Operational  Test  and  Evaluation 
Director,  Program  Analysis  and  Evaluation 
Appendix  9A  Navy  Acquisition  Process 


9.1.3  Summary 

See  below. 


9.2  THE  WORLD  OF  PROGRAM  MANAGEMENT 


9.2.1  The  Role  of  the  Program  Manager 

A  fundamental  Department  of  Defense  (DoD)  policy  is  that  the  acquisition  of  major  weapon  systems 
will  be  directed  by  responsible  managers  under  the  concept  of  program  management. 

The  concept  of  program  management  is  to  provide  centralized  management  authority  over  all  of 
the  technical  and  business  aspects  of  a  program.  The  program  manager’s  role,  then,  is  to  tie  together, 
to  manage,  and  to  direct  the  development  and  production  of  a  system  meeting  performance,  schedule, 
and  cost  objectives  which  are  defined  by  his  Service  and  approved  by  the  Secretary  of  Defense  (SECDEF). 
The  essence  of  the  program  manager’s  role  is  to  be  the  agent  of  the  Service  in  the  management  of 
the  system  acquisition  process  and  to  focus  the  authority  and  responsibility  of  the  Service  for  running 
the  program.  He  has  the  vantage  of  a  large  perspective  of  the  program  and  the  interrelationships  among 
its  elements.  He  must  be  the  major  motive  force  for  propelling  the  system  from  conception  to  completion. 

Recently,  a  panel  of  military  program  managers  examining  their  role  likened  it  to  that  of  the  general 
manager  of  a  small  company.  The  comparison  is  especially  apt.  It  would  be  impossible  to  write  a 
meaningful  position  description  for  that  job.  It  is  equally  impossible  to  write  one  for  the  program 
manager’s  job.  What  the  general  manager  does  is  whatever  is  needed  to  move  the  affairs  of  the  business. 
He  does  one  thing  at  one  time  and  another  thing  at  another  time — whatever  is  most  needed  at  the  moment 
to  achieve  his  objectives.  A  general  manager  is  not  a  “doer”  of  any  job — there  are  other  managers 
charged  with  the  doing.  But  the  general  manager  sees  to  it  that  what  he  wants  is  done,  and  what  he 
wants  is  a  harmony  of  things  done  so  that  his  objectives  are  achieved.  The  role  implies  reliance  on 
others  to  do  the  work;  but  it  also  implies  controlling  and  coordinating  the  work  so  that  no  one  aspect 
dominates  others  to  the  detriment  of  the  harmony  of  the  whole. 

This  touches  upon  what  is  likely  to  be  the  most  important  function  of  the  program  manager:  getting 
people  to  communicate  with  each  other  to  achieve  a  common  understanding  of  the  needs  of  the  program 
and  their  place  in  the  harmony  of  the  total  program  effort. 
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9.2.2  Service  Responsibility 

It  is  an  oversimplification,  but  basically  correct,  to  identify  three  players  and  their  respective  roles: 
the  Service,  the  program  manager,  and  the  Office  of  the  Secretary  of  Defense  (OSD).  The  Service  is 
responsible  for  identifying  its  operational  needs  and  defining  the  new  systems  required  to  meet  those 
needs.  It  is  also  responsible  for  the  formulation  of  the  acquisition  strategy  for  the  orderly  development 
and  production  of  the  systems.  The  program  manager  is  the  agent  of  the  Service  for  the  formulation 
and  execution  of  the  strategy.  OSD  is  the  keeper  of  the  Service  conscience — it  reviews  and  approves 
the  Service  strategy  and  program.  But  the  center  of  systems  acquisition,  authority,  and  responsibility 
lies  in  the  Service — more  specifically,  it  is  the  Service  Secretary. 

Approval  improperly  exercised  means  direction  in  practice.  It  is  possible  to  withhold  approval 
until  the  one  approach  desired  by  the  approving  authority  is  reluctantly  proposed  (or  stumbled  upon) 
by  the  organization  or  person  seeking  the  approval.  That  way  of  exercising  approval  is  directing — 
albeit  obliquely.  He  who  exercises  “approval”  power  in  that  mode  is  seen  to  have  assumed  the  role 
of  directing,  while  perhaps  planning  to  dodge  responsibility  if  things  go  wrong. 

However,  the  role  of  line  and  staff  authority  has  been  delineated  in  the  DoD  Directive 
5000.1 — “Major  System  Acquisitions”  dated  18  January  1977. 

When  a  line  official  above  the  program  manager  exercises  decision  authority  on  program  matters, 

the  decision  shall  be  documented  as  official  program  direction  to  the  program  manager.  The  line 

official  shal  be  held  accountable  for  the  decision.  The  role  of  staffs  as  functional  advisors  does 

not  include  the  authority,  responsibility  or  accountability  for  program  decisions.* 

Approval  means  something  else,  especially  in  the  context  of  OSD’s  role  in  military  program 
management.  It  denotes  a  dictionary  definition  of  the  word  “approval”:  “to  accept  as  satisfactory.” 
That  is  to  say,  it  is  the  Service’s  role  to  formulate  the  system  requirements  and  plan  for  implementation. 
It  is  OSD’s  role  to  accept  the  Service’s  product  as  satisfactory— provided  it  is  consistent  with  major 
policy  objectives.  It  is  also  OSD’s  role  to  evaluate  the  performance  of  the  Services  in  implementating 
the  approved  programs.  But  the  Service  has  the  final  responsibility  for  getting  the  job  done. 

9.2.3  Judgment  and  Flexibility 

The  concept  of  program  management  came  about  because  the  ordinary  way  of  doing  things  was 
not  adequate  for  the  task  of  managing  the  acquisition  of  complex  weapon  systems.  Extraordinary 
management — program-oriented  management — was  essential  if  all  of  the  aspects  of  the  program  were 
to  be  handled  correctly  and  expeditiously. 

To  achieve  this  extraordinary  management,  there  is  another  OSD  policy  which  complements  the 
policy  requiring  program  management:  military  program  managers  should  be  free  to  exercise  judgment 
and  flexibility.  Although  the  program  manager  is  the  agent  of  the  Service,  he  should  operate  in  an 
environment  in  which  he  selects  and  tailors  to  the  specific  needs  of  his  program  those  management 
systems  and  formal  techniques  that  will  help  his  program.  He  should  operate  in  an  environment  conducive 
to  the  exercise  of  judgment.  There  is  no  pet  formula  a  program  manager  can  adopt.  He  must  decide 
for  himself  what  methods,  techniques,  and  systems  he  will  use.  If  the  program  manager  is  responsible 
for  planning,  directing,  and  controlling  a  program,  he  must  have  the  authority  to  get  the  job  done. 


*Dept  of  Defense,  "Major  System  Acquisitions,-'  Directive  5000.1,  18  January  1977,  p  6. 


Stated  another  way,  the  program  manager  is  encouraged  to  adapt  standard  techniques  to  the  peculiar 
requirements  of  his  program.  In  turn,  he  has  a  right  to  expect  that  those  in  the  Services  who  are  going 
to  approve  his  management  plans  and  techniques  will  exercise  their  power  of  approval  properly.  That 
is  to  say,  his  plans  and  techniques  will  be  accepted  as  satisfactory  if  they  comply  with  basic  policy 
directives.  He  has  a  right  to  expect  that  his  plan  will  not  be  judged  by  the  standard  of  meticulous 
compliance  with  innumerable  details  hidden  away  in  regulations,  directives,  instructions,  handbooks, 
manuals,  standards,  specifications,  or  similar  documents. 

What  the  program  manager  has  a  right  to  expect  and  what  in  fact  he  will  be  offered  are  often 
quite  different.  Experienced  program  managers  would  remind  the  new  program  manager  that  often 
one  must  struggle  to  obtain  the  management  flexibility  he  is  supposed  to  be  given.  Higher  authorities, 
and  especially  their  staff  organizations,  tend  to  standardize  their  requirements  and  to  insist  on  the  use 
of  familiar  techniques  and  methods.  Their  initial  disposition  is  to  avoid  changes  and  exceptions  to  the 
general  rule.  Requests  for  deviations  are  rarely  conceded  without  being  pushed  and  sold. 

9.2.4  Functional  Support 

The  use  of  judgment  and  the  exercise  of  flexibility  are  difficult  to  achieve  in  the  environment  of 
military  program  management.  The  most  significant  reason  for  this  is  that  the  operation  of  program 
management  envisions  two  organizational  elements.  In  some  few  cases  the  program  office  is  staffed 
with  all  or  most  of  the  capability  to  perform  the  functional  activities.  In  these  cases  the  program  office 
is  largely  self-sufficient  and  does  not  have  to  rely  on  much  support  from  functional  activities  outside 
of  the  line  authority  of  the  program  manager.  Coordination  is  simplified,  but  the  problems  associated 
with  organizing  and  staffing  the  program  office  are  magnified.  Usually,  however,  there  is  a  small, 
centralized  management  authority  consisting  of  the  program  manager  and  his  program  office.  This 
office  is  served  by  functional  organizations  which  support  the  centralized  authority  and  which  are 
responsible  to  it  for  the  execution  of  assigned  tasks.  This  environment,  where  the  resources  for  doing 
the  work  are  largely  outside  of  the  line  authority  of  the  program  manager,  is  a  natural  source  of  conflict. 

The  practical  fact  is  that  there  are  usually  several  programs  competing  for  the  limited  resources 
of  the  same  functional  organizations.  Those  functional  elements  are  also  supporting  the  normal  activities 
of  their  parent  organizations — the  day-to-day,  nonprogram  activities.  When  personnel  are  not  available 
to  support  all  of  the  demands,  the  program  manager  finds  less  responsiveness  than  he  desires  from 
the  functional  elements.  His  situation  is  made  even  more  difficult  because  the  functional  elements  were 
there  long  before  his  program  started  and  they  plan  to  be  there  long  after  his  program  ends. 

Another  aspect  of  this  problem  is  the  tendency  of  functional  specialists  to  see  their  discipline  as 
the  central  core  of  a  successful  program.  Their  commitment  to  their  specialty  leads  them  to  try  to  dictate 
to  the  program  what  will  or  must  be  done — as  distinguished  from  advising  what  should  be  done.  Further, 
there  is  no  lack  of  regulations  with  which  they  can  bolster  their  claim.  One  of  the  most  difficult  concepts 
to  put  across  to  functional  specialists  is  that  the  program  manager  is  responsible  for  determining  what 
will  be  done,  the  functional  specialist  is  responsible  for  how  it  is  done — the  how  being  his  area  of  expertise. 

There  is  a  natural  tendency  for  the  functional  managers  to  standardize  their  operations  or 
efforts,  to  perform  to  standards,  or  to  build  a  standard  model.  A  program  manager  must,  through 
his  influence,  force  his  functional  areas  to  depart  from  a  standard  and  build  something  that  fits 
in  with  the  other  parts  of  the  project.  Someone  has  to  force  these  people  to  take  action  when  these 
actions  increase  a  functional  manager’s  risk  or  use  his  resources  at  a  greater  rate  than  he  would 
otherwise.  The  program  manager’s  role  is  to  balance  this  risk  over  all  portions  of  the  project. 
Therefore,  he  must  have  authority  to  move  quickly  to  balance  his  risk.* 

•George  A.  Steiner  and  William  G.  Ryan,  Industrial  Project  Management,  the  Macmillan  Company,  1968,  p  29. 
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The  obverse  is  equally  true,  however.  Once  the  government  program  manager  has  obtained  the 
assurance  he  needs,  he  should  relax  his  control  and  concede  to  his  contractors  a  measure  of  freedom 
to  exercise  judgment  and  flexibility  similar  to  that  which  he  seeks  for  himself. 

Problems  with  functional  specialists  are  not  something  new: 

The  expert,  in  fact,  simply  by  reason  of  his  immersion  in  a  routine,  tends  to  lack  flexibility 
of  mind  once  he  approaches  the  margins  of  his  special  theme.  He  is  incapable  of  rapid  adaptation 
to  novel  situations.  He  unduly  discounts  experience  which  does  not  tally  with  his  own.  He  is  hostile 
to  views  which  are  not  set  out  in  terms  he  has  been  accustomed  to  handle.  No  man  is  so  adept 
at  realizing  difficulties  within  the  field  that  he  knows;  but,  also,  few  are  so  incapable  of  meeting 
situations  outside  that  field.  Specialism  seems  to  breed  a  horror  of  unwonted  experiment,  a  weakness 
in  achieving  adaptability,  both  of  which  make  the  expert  of  dubious  value  when  he  is  in  supreme 
command  of  a  situation.* 

The  environment  of  program  management  therefore  places  an  extraordinary  premium  on  talent 
for  leadership  as  distinguished  from  command,  on  persuasion  as  distinguished  from  direction.  The 
environment  requires  an  emphasis  on  informal  authority,  de  facto  authority,  or  influence  as  distinct 
from  power.  One  student  of  program  management  has  described  this  authority  as  derived  in  part  from 
the  program  manager’s  “persuasive  ability,  his  rapport  with  extraorganizational  units,  and  his  reputation 
in  resolving  opposing  viewpoints  within  the  parent  unit  and  between  the  external  organizations.”** 

Persuasion  is  not  the  only  way  to  get  things  done.  One  defense  program  manager  said  that  on 
many  occasions  he  overcame  the  opposition  of  functional  specialists  by  “working  harder  than  they 
did.”  This  program  manager  found  that  he  could  so  overwhelm  a  specialist  with  facts,  figures,  and 
analysis  that  it  became  too  much  of  an  effort  for  the  specialist  to  refute  the  program  manager’s  position. 

The  comments  of  this  program  manager  highlighted  a  point  made  by  several  others  that  there 
js  a  need  for  a  strong  analytical  capability  in  the  program  office  to  coordinate  a  program  whose  parts 
were  organizationally  and  geographically  widely  dispersed.  A  talent  for  analysis  and  ability  to  work 
with  people  were  the  key  criteria  in  their  selection  of  program  office  personnel. 


9.2.5  Engagement  and  Disengagement 

In  common  with  the  way  a  general  manager  must  operate,  the  program  manager  relies  on  others 
to  do  the  work.  But  he  cannot  escape  the  responsibility  for  the  result.  If  he  is  responsible,  he  must 
be  satisfied  that  what  is  done  in  his  program  makes  sense  to  him  and  is  consistent  with  his  plans.  If 
he  cannot  be  persuaded  that  it  is  right  for  his  program,  he  must  direct  it  to  be  done  the  way  he  wants. 

Much  has  been  written  about  the  role  of  industry  and  the  relationship  that  should  obtain  between 
the  defense  program  manager  and  his  industry  counterpart.  Much  has  been  said  about  “disen¬ 
gagement” — getting  out  of  industry’s  hair  and  letting  them  do  the  job  they  have  contracted  to  do. 
The  goal  is  laudable  and,  the  way  it  is  stated,  the  idea  is  entirely  consistent  with  good  management 
concepts.  But  the  ultimate  responsibility  for  a  successful  program  rests  squarely  on  the  Service  and 
on  the  military  program  manager  as  its  agent.  The  program  manager  cannot  disengage  in  any  literal 
sense.  He  must  manage  contracted  work  in  just  the  same  sense  as  he  manages  all  the  other  parts  of 

*  Harold  J.  Laski,  “The  Limitations  of  the  Expert,”  Harper's  Magazine,  December  1930.  Quoted  in  Specialists  and  Gener 
alists,  a  selection  of  readings  by  the  Committee  on  Government  Operations,  U.S.  Senate,  90th  Congress,  2d  Session,  1968,  p  53 

**  David  I  Cleland,  “Project  Management,"  Air-University  Review,  Vol.  XVI,  No.  2,  January-February,  1965.  Reprinted 
in  a  Hook  of  readings  compiled  by  David  I.  Cleland  and  William  R.  King.  Systems.  Organizations.  Analysis.  Management, 
McGraw-Hill  Book  Company,  1969. 
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his  program.  More  precisely,  in  this  case  he  manages  contractor  management  of  his  program.  It  is 
not  a  question  of  whether  he  manages;  it  is  only  a  question  of  how  he  manages — or  mismanages. 

Industry  program  managers  and  government  program  managers  are  agreed  on  this  point: 

It  seems  clear  that  the  Government  program  manager  must  exercise  rather  tight  control  until 
such  time  as  he  is  assured  that  the  industrial  project  manager  has  the  technical  and  managerial 
competence  to  perform  as  required.* 


9.2.6  The  Soft  Sell 

Newly  appointed  program  managers  may  be  dismayed  to  discover  that  there  is  less  than  complete 
and  enthusiastic  support  for  their  programs  within  their  Service  and  OSD.  Every  weapon  system  competes 
with  ail  the  others  for  limited  resources,  and  competition  is  especially  fierce  in  periods  of  tight  budgets. 
At  every  level  in  the  hierarchy,  commanders  and  staff  personnel  are  confronted  by  demands  from  program 
and  functional  managers  for  far  more  money  than  is  available  or  can  reasonably  be  obtained.  Budget 
recommendations  and  decisions  must  be  made  that  will  inevitably  favor  some  programs  over  others. 

The  program  manager  who  has  done  his  homework  and  has  kept  key  people  informed  about  his 
system’s  programs  and  progress  will  improve  the  odds  that  funds  for  his  program  will  not  be  reduced. 
We  are  not  suggesting  that  a  program  manager  affect  a  hard-sell  stance  or  that  he  patrol  corridors 
to  buttonhole  unwary  staff  people.  What  we  are  suggesting  is  that  a  program  manager  should  be  attuned 
to  the  information  needs  and  biases  of  the  people  who  influence  budget  decisions.  This  implies  a  kind 
of  low-key  salesmanship — of  the  soft-sell,  helpful  variety. 

One  of  the  program  manager’s  greatest  sources  of  authority  involves  the  manner  in  which 
he  builds  alliances  in  his  environment— with  his  peers,  associates,  superiors,  subordinates,  and 
other  interested  parties.  The  building  of  alliances  supplements  his  legal  authority;  it  is  the  process 
through  which  the  project  manager  can  translate  disagreement  and  conflict  into  authority  (or 
influence  power)  to  make  his  decisions  stand.  Sometimes  the  power  and  control  of  the  project 
manager  represents  a  subtle  departure  from  his  legal  authority. 

The  program  manager  must  keep  in  touch  with  what  is  going  on  above  him.  He  has  to  be  aware 
of  what  is  expected  of  him  by  higher  authority — both  in  his  Service  and  at  the  OSD  level.  He  should 
know  the  typical  questions  being  asked  at  major  program  review  points,  and  he  should  be  aware  that 
these  requirements  for  information  by  higher  authority  are  constantly  changing. 

Program  managers  speak  at  length  on  the  need  to  instill  confidence  in  superiors.  This  confidence 
is  a  foundation  of  rapport  with  superiors  which,  in  turn,  is  one  of  the  main  sources  of  the  program 
manager’s  authority.  When  it  is  obvious  to  functional  managers  supporting  the  program  that  the  program 
manager  has  this  rapport  with  his  superiors,  he  will  not  need  to  rely  as  much  on  formal  authority. 
One  of  the  ways  this  confidence  can  be  instilled  is  by  demonstrating  a  knowledge  of  the  program  in 
the  widest  context.  Knowledge  of  the  program  must  embrace  the  threat,  the  direction  in  which  the 
threat  is  moving,  other  systems  in  the  inventory  that  address  the  threat,  program  schedules,  costs, 
technology — in  short,  everything  important  about  the  program. 


•Steiner  and  Ryan,  op.  cit.,  p  125. 
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9.3  DEFENSE  COMMUNIQUE  ON  DEFENSE  ACQUISITION  MANAGEMENT 


(The  statement  by  Dr.  Richard  D.  Delauer,  Under  Secretary  of  Defense  for  Research  and  Engineering 
before  the  Senate  Armed  Services  Committee,  November  16,  1983  on  Defense  Acquisition  Management 
offers  valuable  guidelines.) 

9.3.1  Management  Philosophy 

Our  approach  to  management  is  one  in  which  we  strive  for  a  proper  balance  between  policy 
formulation  and  resource  allocation  on  one  hand,  and  decentralized  program  execution  on  the  other. 
In  order  for  this  concept  to  be  effective,  it  is  imperative  to  have  the  necessary  management  oversight 
to  ensure  that  policies  and  plans  are  being  implemented.  Since  the  Carlucci  initiatives  were  adopted 
over  two  years  ago,  the  Department  has  made  great  progress  in  implementing  this  philosophy.  Within 
the  Department  we  have  made  significant  progress  toward  implementing  a  more  efficient  and  effective 
management  focus. 

Before  I  describe  the  major  organizations  and  offices  which  are  involved  in  managing  the  acquisition 
process  within  the  Department,  let  me  outline  briefly  the  process.  Our  defense  requirements  are  established 
each  year  by  the  Secretary  in  cooperation  with  the  Services  and  the  Joint  Chiefs  [of  Staff  (JCS)]  through 
defense  guidance.  The  Services  submit  a  5-year  plan  called  the  Program  Objective  Memorandum  (POM) 
to  the  Secretary  based  upon  defense  guidance.  The  Service  plans  are  analyzed  for  completeness  and 
consistency  with  our  basic  policies.  Any  inconsistencies  result  in  issues  which  are  brought  before  the 
Defense  Resources  Board  (DRB). 

The  DRB,  chaired  by  the  Deputy  Secretary,  is  the  major  decisionmaking  body  in  the  Department’s 
resource  allocation  process.  Participative  management  governs  the  Board’s  activities  since  high  level 
representatives  from  all  major  DoD  components,  including  the  Service  secretaries  and  JCS  participate 
in  DRB  decisions.  The  focus  of  DRB  attention,  however,  is  upon  issues  where  resources  do  not  fulfill 
policy  objectives. 

During  this  year’s  program  review,  issues  settled  by  the  DRB  accounted  for  about  3  percent  of 
the  total  funding  requests  submitted  in  this  year’s  POM.  Moreover,  this  small  proportion  represented 
only  issues  of  highest  priority  where  the  DRB  decided  that  the  initially  proposed  resource  were  not 
appropriate  for  the  required  task. 

As  a  partner  in  defense  acquisition  management,  Congress  shares  the  responsibility  to  participate 
in  policy  formulation  and  implementation  oversight.  However,  it  seems  that  over  the  years,  Congress 
has  digressed  from  an  oversight  role  in  which  it  would  participate  in  the  establishment  of  policy  objectives 
and  measure  progress  toward  achieving  the  policy  goals.  Unfortunately,  congressional  oversight  has 
become  far  too  detailed  to  provide  policy  makers  or  the  public  with  a  coherent  view  of  our 
accomplishments  or  our  needs.  The  solution  to  this  problem  comes  down  to  asking  the  right  questions, 
receiving  the  appropriate  information,  and  intervening  only  when  things  go  off  track. 

For  example,  we  should  be  asking  questions  about  our  objectives  on  the  basis  of  mission  areas. 
"Where  are  we  going,”  we  should  ask,  “in  strategic  forces;  air,  sea,  and  land  mobility;  conventional 
forces,  etc.?”  We  have  worked  this  problem  pretty  well  in  the  areas  of  strategic  offensive  forces  and 
in  air  mobility  for  example.  We  haven’t  yet  done  well  at  all.  however,  in  sea  and  land  mobility  or 
in  the  general  area  of  conventional  forces. 

We  have  observed,  however,  congressional  oversight  in  practice  has  become  more  of  an  annual 
exercise  in  line  item  management.  Due  to  the  parochial  interests  of  constituencies  and  the  increase  in 


size  and  diversity  of  congressional  staffs,  every  year  we  must  assume  a  “prevent  defense’’  on  a 
programmatic  basis.  What  is  needed  is  increased  awareness  of  our  mission  goals  and  our  progress  toward 
their  achievement — not  whether  this  or  that  program  needs  to  be  adjusted. 

The  information  which  is  provided  to  Congress  needs  to  reflect  the  oversight  function  which  I 
have  described.  For  example,  the  Selected  Acquisition  Reports  (SARs)  to  Congress  need  some 
modification  in  order  to  be  more  meaningful  for  effective  oversight.  First,  I  believe  SARs  should  be 
based  on  a  5-year  planning  period  rather  than  for  the  total  program  inventory  objective  as  is  presently 
the  case.  The  problem  is  that  adjustments  to  these  outyear  inventory  objectives  change  program  costs. 
The  result  looks  like  poor  management  rather  than  a  reflection  of  the  uncertainties  of  these  outyear 
projections.  To  make  valid  estimates  of  the  inventory  objectives  of  our  major  systems  beyond  a  5-year 
period  and  have  them  be  meaningful  is  almost  impossible.  We  recognize  the  need  to  have  planning 
figures  for  the  total  program,  but  the  uncertainty  in  these  numbers  must  also  be  acknowledged. 

In  addition,  congressional  oversight  should  be  practiced  on  a  “by-exception”  basis  where  only 
the  major  problems  are  addressed.  Recent  changes  brought  about  by  the  Nunn-McCurdy  Act  have 
increased  the  reporting  requirement  for  major  systems  by  almost  50  percent.  This  additional  requirement 
necessitates  thousands  of  additional  manhours  of  preparation  and  review  for  a  variety  of  systems,  many 
of  which  are  not  sufficiently  mature  to  establish  a  meaningful  baseline.  Moreover,  the  programmatic 
structure  of  the  SAR  reinforces  the  line  item  management  mentality  of  which  I  have  already  spoken. 

Everyone  agrees  that  what  is  needed  most  of  all  in  the  acquisition  process  is  greater  stability.  Yet, 
each  year  we  observe  that  hundreds  of  programs  are  adjusted  by  the  Congress  in  accordance  with 
undefined  priorities.  The  following  year,  we  must  return  to  Congress  to  attempt  to  get  essential  programs 
back  on  track.  As  a  result  of  our  adjustment  and  other  inefficiencies,  it  now  takes  from  12  to  17  years 
to  complete  the  acquisition  process  for  most  major  items.  If  we  could  simply  recognize  the  instability 
which  we  introduce  into  the  acquisition  process  and  the  consequent  cost  (both  in  dollars  and  national 
security),  we  will  have  achieved  a  major  step  forward.  We  are  improving  stability  through  many  of 
our  initiatives  such  as  multiyear  procurement,  economic  production  rates,  and  realistic  budgeting. 
However,  much  more  needs  to  be  done.  I  strongly  endorse  the  Secretary’s  recommendation,  for  example, 
that  the  Congress  adopt  a  2-year  budget  cycle  to  help  alleviate  the  instability  problem.  I  believe  it  would 
reduce  the  average  time  needed  for  completion  of  the  acquisition  process,  and  would  also  mean  signifi¬ 
cant  cost  savings  in  the  long  run. 

9.3.2  Funding  Distinctions 

Another  contributing  factor  to  inefficient  management  which  constrains  the  acquisition  process 
is  the  artifical  distinction  which  is  made  among  various  types  of  funding.  The  acquisition  process  embraces 
exploratory  research,  engineering  development,  manufacturing  assembly,  and  deployment.  To  assume 
that  these  phases  can  be  defined  and  funded  in  a  rigid  manner  (RDT&E  and  production)  is  nonsense. 
It  is  artificial  and  expensive.  Resources  should  be  applied  as  is  necessary  to  do  the  job  in  the  most 
efficient  way.  Line  managers  who  are  close  to  the  program  should  retain  the  flexibility  to  make  these 
judgments  and  act  accordingly.  If  we  could  remove  this  artificial  distinction  in  characterizing  funds, 
I  believe  we  could  save  money  and  shorten  the  process  as  much  as  10  to  20  percent. 

We  are  now  trying  to  add  more  emphasis  to  the  critical  transition  period  from  R&D  to 
manufacturing.  We  are  establishing  a  new  DoD  directive  to  enhance  our  attention  to  this  problem, 
and  I  am  looking  at  ways  to  strengthen  the  production  and  manufacturing  areas  within  DoD.  I  think 
we  need  to  beef-up  the  R&E  (research  and  engineering)  organization  in  this  area.  We  may  well  need 
additional  manpower  spaces  to  devote  to  this  problem. 
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Another  subject  which  causes  us  great  concern  is  the  present  definition  ot  competitive  procurement. 
The  way  we  keep  score  is  very  misleading  to  the  uninitiated  and  we  need  to  redefine  the  terms.  Practically 
all  of  our  programs  are  initially  competitive,  and  we  examine  the  acquisition  strategy  in  great  detail 
to  seek  opportunities  for  dual  sourcing.  We  particularly  try  to  pursue  competition  at  the  subcontractor 
and  vendor  levels.  Because  our  procurement  programs  are  funded  on  an  annual  basis,  however,  we 
do  not  get  credit  (in  a  competitive  sense)  for  follow-on  buys  for  programs  which  reaped  the  benefits 
of  competition  earlier.  It  makes  no  sense,  for  example,  to  develop  a  second  source  for  an  F- 15  or  F- 16 
aircraft  at  this  point.  These  programs  experienced  vigorous  competition  early  in  their  development. 
The  same  is  true  for  most  of  our  major  programs.  This  is  just  another  area  where  we  suffer  bad  press 
for  something  which  is  really  not  a  problem. 

9.3.3  Organization  of  DoD  Acquisition  Management 

A  number  of  organizations  and  offices  play  a  variety  of  roles  in  the  defense  acquisition  process 
within  the  context  of  the  management  philosophy  I  have  described.  Let  me  briefly  summarize  some 
of  the  more  important  and  discuss  their  respective  roles:  I  have  already  mentioned  the  DRB  and  the 
vittl  role  which  it  plays.  Of  course,  there  is  also  the  Under  Secretary  of  Defense  (Research  and 
Engineering/Defense  Acquisition  Executive).  As  the  principal  advisor  to  the  Secretary  on  scientific 
and  acquisition  issues,  I  exercise  oversight  on  scientific  and  technical  matters  of  basic  applied  research, 
and  the  development  and  acquisition  of  our  weapons  systems.  It  is  my  duty  to  ensure  that  activities 
in  these  areas  adhere  to  departmental  policies  and  national  security  objectives.  I  participate  in  the  review 
and  evaluation  of  requirements  and  priorities  and  make  certain  that  our  programs  are  designed  to 
accommodate  operational  requirements.  It  is  also  my  responsibility  to  follow  up  and  evaluate  programs 
for  carrying  out  approved  acquisition  policies  and  standards.  Through  my  participation  in  the  planning, 
programming,  and  budgeting  system,  I  review  proposed  programs  and  resources  and  recommend  resource 
allocations  in  accordance  with  national  security  policy  and  priorities.  Finally,  as  Defense  Acquisition 
Executive  (DAE),  I  serve  as  the  chairman  of  the  Defense  Systems  Acquisition  Review  Council  (DSARC) 
which  exercises  oversight  and  advises  the  Secretary  on  the  management  process,  policies,  and  procedures 
for  acquiring  our  major  programs. 

Though  the  list  of  my  duties  and  other  organizational  commitments  could  go  on,  let  me  just 
emphasize  an  important  aspect  of  my  job  about  which  some  of  the  members  of  the  [Senate  Armed 
Services]  Committee  have  indicated  concern.  The  concern  involves  an  apparent  conflict  of  interest  between 
my  role  as  manager  of  research  and  engineering  and  as  the  director  of  acquisition. 

Quite  frankly,  for  a  variety  of  reasons,  I  see  no  conflict  at  all.  First,  there  is  considerable  overlap 
between  the  functions  of  development  and  production  which  requires  constant  oversight  in  order  to 
manage  the  transition  effectively.  Development  and  production  cannot  be  neatly  separated.  My  office 
possesses  the  technical  and  management  skills  to  exercise  effective  oversight  in  this  area. 

Second,  by  virtue  of  my  role  in  the  planning,  programming,  and  budgeting  process,  as  well  as 
my  position  as  DAE,  I  can  help  to  ensure  that  consistency  exists  between  our  policies  and  requirements 
and  the  systems  which  we  acquire.  Although  the  ultimate  responsibility  regarding  this  fundamental 
objective  is  shared  by  the  Secretary,  the  President,  and  the  Congress,  the  important  pieces  of  the  puzzle 
must  begin  to  fit  together  at  some  point  in  the  management  process.  My  duties  assure  that  this  process 
can  appropriately  be  undertaken  in  the  office  of  the  Under  Secretary  for  Research  and  Engineering. 

Another  important  element  within  the  Department's  oversight  function  was  instituted  by  the  Secretary 
shortly  after  he  took  office.  Each  week  on  a  rotating  basis,  the  service  secretaries  report  to  the  Secretary 
on  one  or  two  major  programs  to  discuss  problems  and  possible  solutions  For  some  of  our  more  critical 
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systems,  program  personnel  provide  regular  reports  directly  to  the  Secretary  on  a  biweekly  or  monthly 
basis.  These  Secretarial  Performance  Reviews  are  an  essential  part  of  the  management  philosophy  I 
described  at  the  outset  of  my  statement:  high  level  oversight  of  high  priority  major  issues. 

No  discussion  of  acquisition  management  is  complete  without  recognizing  the  hard  work 
accomplished  by  the  various  program  offices,  the  buying  commands,  the  Joint  Logistics  Commanders, 
and  the  Defense  Logistics  Agency.  The  personnel  employed  in  these  organizations  perform  an  outstanding 
job  of  executing  the  programs,  policies,  and  procedures  which  confront  them  daily.  We  are  doing  our 
best  to  make  their  jobs  easier  through  various  acquisition  reforms  such  as  reducing  the  number  of 
directives  and  carefully  screening  contractual  data  requirements.  We  maintain  close  contact  with  the 
Joint  Logistics  Commanders  to  ensure  that  these  and  other  important  acquisition  reforms  which  affect 
the  buying  community  are  fully  implemented. 

9.3.4  Oversight 

The  Inspector  General  (IG)  is  another  important  participant  in  the  acquisition  management  oversight 
function.  The  IG  performs  essential  audits  and  reviews  to  ensure  that  waste,  fraud,  and  abuse  are  purged 
from  the  system.  In  addition,  we  receive  valuable  insights  on  how  to  improve  the  acquisition  management 
process  from  the  Council  on  Integrity  and  Management  Inprovement  (CIMI)  which  is  chaired  by  the 
Deputy  Secretary.  The  Council  is  a  high  level  group  which  meets  to  explore  new  ideas  and  opportunites 
for  progress  in  management  on  a  DoD-wide  basis.  Specific  management  issues  are  regularly  considered 
and  the  Council  makes  recommendations  to  the  Secretary  for  his  resolution.  The  Defense  Science  Board 
is  another  important  organization  which  is  external  to  the  Department  and  which  draws  together 
management  ideas  from  key  elements  within  DoD,  industry,  academia,  and  the  scientific  community. 

Finally,  I  must  recognize  my  “right-hand  woman”  who  serves  as  the  Deputy  for  Acquisition 
Management.  Mary  Ann  Gilleece  is  my  principal  advisor  on  acquisition  matters  and  is  the  primary 
policy  maker  for  all  DoD  procurement  and  production  policy.  She  is  responsible  for  implementing 
federal  policies  on  acquisition,  the  formulation  and  revision  of  defense  acquisition  regulations, 
specifications  and  standards,  and  other  policies.  Ms.  Gilleece  is  also  my  principal  advisor  in  deliberations 
of  the  DSARC  on  acquisition  matters,  including  the  implementation  of  acquisition  improvement  program 
initiatives. 

The  members  of  this  committee  are  doubtless  aware  of  many  of  the  important  strides  we  have 
made  in  implementing  various  initiatives  of  the  [Acquisition  Improvement  Programs  (AIP)].  I  won’t 
review  in  detail  the  progress  we  have  made  through  the  good  efforts  of  Ms.  Gilleece  and  many  others, 
but  let  me  highlight  some  changes  we  have  initiated  within  the  DSARC  process  which  make  our 
management  more  efficient  and  effective. 

9.3.5  DSARC  (Defense  Systems  Acquisition  Review  Council] 

[DSARC  is  an  OSD-level  recommending  body  that  is  in  the  decisionmaking  process,  but  does  not 
provide  funds.)  First,  as  I  stated  at  the  outset,  we  have  instituted  controlled  decentralization  throughout 
the  process.  The  Service  secretaries,  for  example,  now  serve  as  full  members  of  the  DSARC  and  contribute 
to  policy  formulation  and  program  execution.  [Other  members  of  the  council  include  the  USDRE 
chairman,  USD  (Policy),  ASD  (MI&L),  ASD  (Q,  DIR  (PA&E),  chairman  of  the  JCS,  and  DIR  (OT&E).] 
At  the  same  time,  we  have  streamlined  the  acquisition  process  in  order  to  accelerate  the  results  and 
avoid  micromanagement.  The  front  end  review  of  new  programs  is  now  done  during  the  POM  review 
process  to  ensure  that  the  new  starts  are  both  required  anu  at  fordable. 
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In  addition,  by  revising  the  threshold  which  defines  a  major  system  to  $200  million  in  R&D  and 
$1  billion  in  procurement,  we  have  ensured  that  high  level  management  attention  is  placed  on  programs 
of  major  significance.  Major  milestone  decisions  on  10  programs  were  delegated  back  to  the  services 
when  we  instituted  this  change  in  1981,  and  milestone  decisions  for  an  additional  10  or  so  programs 
have  been  precluded  from  the  DSARC  process  since  that  time.  Partly  as  a  consequence,  DSARC  activity 
during  the  last  3  years  has  decreased  significantly  from  12  meetings  in  1981  to  5  meetings  thus  far  in 
1983.  I  should  add  that  much  of  what  might  have  required  a  DSARC  in  the  past  has  been  replaced 
by  much  less  formal  program  reviews  which  focus  on  specific  major  problem  areas.  Fewer  prereviews 
and  reduced  documentation  has  been  a  major  benefit. 

All  major  programs  proceed  through  the  DSARC  process  to  milestone  II  when  a  program  is  baselined 
at  full-scale  development.  If  a  program  stays  within  baseline  objectives,  the  final  production  decision 
is  made  at  the  appropriate  management  level,  i.e.,  the  Service.  If  a  program  exceeds  its  baseline  objectives, 
however,  it  remains  subject  to  further  DSARC  review  before  a  production  go-ahead  can  be  given. 

At  each  DSARC  milestone  we  review  acquisition  strategy  and  examine  the  potential  for  competition, 
including  the  possibility  for  dual  sourcing.  We  ensure  that  support  and  readiness  are  given  proper 
consideration,  and  determine  whether  a  program  meets  the  criteria  to  become  a  multiyear  candidate. 
In  addition,  greater  emphasis  is  now  being  placed  in  the  DSARC  on  improved  production  engineering 
preparation  to  ease  the  difficult  transition  from  development  to  manufacturing.  We  also  examine  other 
AIP  initiative  areas  such  as  the  potential  to  apply  a  preplanned  product  (P3)  improvement  approach. 
In  addition,  the  Cost  Analysis  Improvement  Group  (CAIG)  examines  cost  estimates  at  each  milestone 
in  order  to  ensure  realistic  budget  estimates.  In  short,  each  DSARC  meeting  is  an  opportunity  to  confirm 
that  the  acquisition  policies  we  have  adopted  are  being  considered  and  executed  properly  at  lower  levels 
of  management.  The  DSARC  process  is  alive  and  well  and  more  effective  than  ever  as  a  means  to 
improve  our  management  of  the  acquisition  process. 


9.3.6  Other  Management  Issues 

The  Committee  has  also  indicated  its  interest  in  the  area  of  joint  service  program  management 
as  a  means  to  increase  efficiency  and  otherwise  reduce  the  aggregate  costs  of  weapons  systems 
development  and  procurement.  Although  the  benefits  of  joint  service  programs  can  be  attractive, 
expensive  failures  have  occurred  too  often  in  the  past  and  counsel  caution  as  we  proceed  forward. 
A  balanced  approach  is  needed  whereby  specific  service  requirements  are  carefully  examined  to  ensure 
compatibility  before  proceeding  on  a  joint  basis.  Our  basic  goal  is  to  increase  cross-service  coordination 
in  the  development  and  use  of  the  systems  and  technology  of  similar  purposes  to  obtain  maximum 
performance  at  minimum  cost. 

Some  recent  examples  where  we  are  making  important  progress  involve  joint  activities  in  specific 
mission  areas.  The  Navy  and  the  Air  Force,  for  example,  have  achieved  important  progress  in 
coordinating  their  efforts  regarding  Fleet  air  defense  and  sea  control.  The  decision  to  designate  a  number 
of  B-52s  to  support  the  Navy’s  sea  control  mission  is  a  good  example  of  the  benefits  of  improved  cross- 
service  coordination  and  management.  Similar  efforts  are  being  conducted  in  the  deep  interdiction  mission 
area  where  hardware  development  as  well  as  employment  doctrine  and  concepts  are  being  examined 
on  a  joint  basis.  We  are  confident  that  the  results  of  this  joint  effort  will  produce  an  effective  means 
to  solve  the  second  echelon  problem  in  the  most  economical  way  possible. 


9.3.7  Joint  Requirements  and  Management  Board 

We  are  considering  some  important  steps  to  institutionalize  our  approach  and  practices  concerning 
joint  service  programs.  As  a  result  of  a  study  conducted  this  past  summer  by  the  Defense  Science  Board, 
a  Joint  Requirements  and  Management  Board  comprising  the  service  Vice-Chiefs,  the  Director  of  the 
Joint  Staff,  and  OSD  components  is  being  contemplated.  The  Board’s  primary  purpose  would  be  to 
provide  a  rigorous  review  of  the  requirements  process  and  identify  those  with  potential  for  joint  service 
applicability.  The  Board  would  provide  a  more  systematic  way  of  examining  the  possibilities  for  joint 
programs  than  we  have  had  in  the  past. 

Another  area  of  particular  interest  these  days  concerns  the  acquisition  of  spare  parts;  the  horror 
stories  have  created  understandable  concern  among  all  of  us.  First,  may  I  point  out  that  our  system 
of  management  works.  Employees  of  the  Department  discovered  and  reported  the  problem,  and  the 
management  of  the  Department  has  taken  corrective  measures.  Secretary  Weinberger’s  10  points  program 
on  spare  parts  procurement  reform  has  already  revised  much  of  the  way  in  which  we  do  business. 

Contracts  to  purchase  spares  are  being  written  to  ensure  that  spares  are  purchased  competitively 
to  the  maximum  extent  possible.  Steps  are  being  taken  to  ensure  that  manufacturers  of  parts  are  identified 
and  that  complete  technical  data  packages  become  available  so  that  we  appoint  at  the  mercy  of  the 
prime  contractor  in  the  future.  The  bidders  for  the  new  fighter  engine  (Pratt  and  Whitney  and  General 
Electric)  are  being  required  to  submit  plans  to  show  how  they  will  develop  two  or  more  qualified 
subcontractors  who  would  remain  available  for  production  of  the  30  replenishment  spares  with  the 
highest  procurement  value.  Since  those  high  value  parts  comprise  about  80  percent  of  the  value  of 
the  engine,  we  can  focus  our  efforts  on  gaining  competition  for  the  remaining  20  percent. 

In  addition,  we  are  building  incentives  and  disincentives  into  the  spares  management  process.  The 
Air  Force  sergeant  who  reported  a  case  of  pricing  abuse  was  recently  awarded  $1,100  (the  amount 
the  Air  Force  was  being  charged  for  the  spare  part).  Where  an  employee’s  negligence  has  contributed 
to  excessive  prices,  we  are  taking  disciplinary  action.  Where  industry  is  at  fault,  we  will  use  every  legal 
means  to  obtain  refunds  in  case  of  overcharge. 

The  Department  and  the  Congress  are  partners  in  the  process  of  acquisition  management.  I  have 
attempted  to  describe  just  a  few  of  the  major  characteristics  and  objectives  of  the  departmental  acquisition 
organization  and  management  for  you  today.  However,  our  organization  and  its  efforts  cannot  be 
successful  unless  each  member  of  this  Committee  and  the  Congress  as  a  whole  understands  their  purpose 
and  potential  benefit.  Moreover,  each  member  of  Congress  should  simultaneously  consider  the 
appropriate  management  role  of  the  legislative  branch  in  the  acquisition  process.  Are  national  or  parochial 
interests  being  served  when  Congress  votes  in  a  particular  way?  Is  national  security  truly  enhanced? 
When  the  Congress  votes  to  reduce  funds  for  a  particular  program,  do  the  members  understand  the 
cost  impact  for  the  future?  Certainly,  the  answers  to  these  questions  vary  from  member  to  member. 
The  concepts,  however,  are  important  to  bear  in  mind  lest  we  seek  inappropriate  micro  solutions  to 
problems  which  are  truly  macro  in  nature. 

[Figures  9.1  and  9.2  present  the  life  cycle  of  typical  major  system  acquisitions  and  life-cycle  costing 
for  those  acquisitions.] 


9.4  THE  DEPARTMENT  OF  DEFENSE  ORGANIZATIONAL  STRUCTURE 


9.4.1  Introduction 


system  acquisitions. 


Life-cycle  costing  in  system  acquisition 


9.4.1. 1  The  Department  of  Defense  (DoD)  (DoD  Directive  5100.1).  DoD  is  responsible  for  providing 
the  military  forces  needed  to  deter  war  and  protect  the  security  of  the  United  States.  The  major  elements 
of  these  forces  are  the  Army,  Navy,  Air  Force,  and  Marine  Corps.  Under  the  President,  who  is  also 
Commander-in-Chief,  the  Secretary  of  Defense  exercises  direction,  authority,  and  control  over  the 
Department  which  includes  the  Office  of  the  Secretary  of  Defense,  organization  of  the  Joint  Chiefs 
of  Staff,  three  military  departments,  nine  unified  and  specified  commands,  the  DoD  Inspector  General, 
twelve  defense  agencies,  and  six  OSD  field  activities  (Figure  9.3). 

In  order  to  support  the  forces  that  DoD  provides,  the  acquisition  of  major  systems  is  an  inevitable 
necessity.  Some  brief  definitions  should  be  helpful  for  considering  the  items  in  this  section.  A  major 
system  exhibits  the  following  characteristics: 

a.  It  has  been  determined,  at  the  discretion  of  the  agency’s  head  (e.g.,  the  Secretary  of  Defense), 
to  be  critical  to  fulfilling  the  agency’s  mission. 

b.  It  entails  the  allocation  of  relatively  large  resources. 

c.  It  warrants  special  management  attention. 

Besides  having  the  characteristics  noted  above,  a  DoD  major  system  has  been  designated  as  such  based 
on  the  following  considerations: 

a.  Risk,  need,  or  other  item  of  SECDEF  interest 

b.  Joint  service  or  multinational  acquisition 

c.  Estimated  RDT&E  and  procurement  funds 

d.  Estimated  operations,  maintenance,  and  support  manpower  requirements 

e.  Congressional  interest. 

DoD  Directive  5000. 1  describes  major  systems  and  DoD  major  systems  in  greater  detail.  The  directive 
emphasizes  the  following  themes: 

a.  Flexibility  (tailored  acquisition  strategy) 

b.  Minimize  time  to  field  new  capability 

c.  Balance  between  cost  and  effectiveness 

d.  Linkage  with  PPBS 

e.  Maximize  collaboration  with  allies 

f.  Integration  of  support  considerations  into  process 

g.  Integration  of  threat  considerations  into  process 

h.  Decentralization: 

SECDEF— two  milestone  decisions  on  major  systems  services— all  other  management  on 
major  systems. 

The  directive  addresses  the  management  of  nonmajor  programs  as  well.  “The  management  principles 
and  objectives  in  this  Directive  (DoDD  5000. 1)  shall  also  be  applied  to  the  acquisition  of  defense  systems 
not  designated  as  major.”  This  particularly  includes  tailored  application  and  decentralized  management 
responsibility. 

Examples  of  current  major  systems  are  listed  in  Table  9. 1 . 

9.4.1.2  The  Office  of  the  Secretary  of  Defense  (OSD).  OSD  is  the  principal  staff  element  of  the 
Secretary  in  the  exercise  of  policy  development,  planning,  resource  management,  fiscal,  and  program 
evaluation  responsibilities  (Figure  9.4).  OSD  includes  the  immediate  offices  of  the  Secretary  and  Deputy 
Secretary  of  Defense,  Under  Secretary  of  Defense  for  Policy,  Under  Secretary  of  Defense  for  Research 
and  Engineering  (USDRE),  Assistant  Secretaries  of  Defense,  General  Counsel,  Assistants  to  the  Secretary 
of  Defense,  and  such  other  staff  offices  as  the  Secretary  establishes  to  assist  in  carrying  out  assigned 
responsibilities. 
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Figure  9.3.  The  Department  of  Defense. 
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Figure  9.4.  Office  of  the  Secretary  of  Defense. 
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The  organizational  struture  of  USDRE  is  presented  in  Figures  9.5  and  9.6.  The  USDRE  has  three 
major  roles: 

a.  Principal  advisor  to  the  SECDEF  for  scientific  and  technical  matters 

b.  Defense  acquisition  executive  (DSARC  chairman) 

c.  U.S.  representative  to  the  NATO  Conference  of  National  Armament  Directors  (CNAD). 

9.4.1.3  The  Military  Departments  (DoD  Directive  5100.1).  These  include  the  Departments  of  the 
Army,  Navy,  and  Air  Force  (the  Marine  Corps  is  a  part  of  the  Department  of  the  Navy).  Each  military 
department  is  separately  organized  under  its  own  Secretary  and  functions  under  the  direction,  authority, 
and  control  of  the  Secretary  of  Defense.  The  military  departments  are  responsible  for  organizing,  training, 
supplying,  and  equipping  forces  for  assignment  to  the  unified  and  specified  commands  (U/S  Commands). 

9.4.1.4  The  Joint  Chiefs  of  Staff  (JCS)  (DoD  Directive  5100.1).  The  JCS  are  the  principal  military 
advisors  to  the  Secretary  of  Defense  as  well  as  to  the  President  and  the  National  Security  Council. 
Members  of  the  Joint  Chiefs  of  Staff,  other  than  the  Chairman,  are  the  senior  military  officers  of 
their  respective  Services  and  are  responsible  for  keeping  the  Secretaries  of  the  military  departments 
fully  informed  on  matters  considered  or  acted  upon  by  the  Joint  Chiefs  of  Staff. 

9.4.1.5  The  Armed  Forces  Policy  Council  (AFPC)  (DoD  Directive  5105.3).  The  AFPC  advises  the 
Secretary  of  Defense  on  matters  of  broad  policy  relating  to  the  Armed  Forces  and  such  other  matters 
as  the  Secretary  may  direct.  Its  members  report  regularly  on  important  matters  under  their  cognizance 
which  are  of  interest  to  the  Department  of  Defense.  In  addition  to  members  identified  below,  such 
other  officials  of  the  Department  of  Defense,  and  other  departments  and  agencies  in  the  Executive 
Branch  as  may  be  designated  by  the  Secretary  of  Defense,  are  invited  to  attend  appropriate  meetings 
of  the  AFPC.  Council  membership  is  as  indicated  below: 

Secretary  of  Defense,  Chairman 
Deputy  Secretary  of  Defense 
Secretaries  of  the  Military  Departments 
Chairman,  Joint  Chiefs  of  Staff 
Under  Secretaries  of  Defense 
Chief  of  Staff,  Army 
Chief  of  Naval  Operations 
Chief  of  Staff,  Air  Force 
Commandant,  Marine  Corps. 

9.4.1.6  The  Unified  and  Specified  Commands  (U/S  Commands)  (DoD  Directive  5100.1).  The  U/S 
Commands  are  responsible  to  the  President  and  the  Secretary  of  Defense  for  the  accomplishment  of 
the  military  missions  assigned  to  them.  Combatant  units  of  the  military  departments  are  assigned  to, 
and  under  the  operational  command  of,  Commanders  of  unified  and  specified  commands.  Unified 
commands  are  composed  of  assigned  components  of  two  or  more  Services.  They  include  the  European 
Command,  Pacific  Command,  Atlantic  Command,  Southern  Command,  Readiness  Command,  and 
Central  Command.  Specified  commands  are  usually  composed  of  forces  from  one  Service,  but  may 
include  units  and  have  representation  from  other  Services.  They  include  the  Aerospace  Defense 
Command,  Strategic  Air  Command,  and  Military  Airlift  Command.  The  military  chain  of  command 
runs  from  the  President  to  the  Secretary  of  Defense  and,  through  the  Joint  Chiefs  of  Staff,  to  the 
Commanders  of  unified  and  specified  commands.  Orders  to  these  Commanders  are  issued  by  the 
President  or  the  Secretary  of  Defense,  or  by  the  Joint  Chiefs  of  Staff  by  authority  and  direction  of 
the  Secretary  of  Defense. 
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Figure  9.5.  Office  of  the  Under  Secretary  of  Defense  for  Research  and  Engineering. 


Under  Secretary  of  Defense  —  Research  &  Engineering. 


9.4.1. 7  The  Inspector  General  (IG)  of  the  Department  of  Defense  (DoD  Directive  5106.1).  The  IG, 
under  the  provisions  set  forth  by  Public  Law  95-452,  serves  as  an  independent  and  objective  official 
in  the  Department  of  Defense  who  is  responsible  for  conducting,  supervising,  monitoring,  and  initiating 
audits  and  investigations  relating  to  programs  and  operations  of  the  Department  of  Defense.  The 
Inspector  General  provides  leadership  and  coordination  and  recommends  policies  for  activities  designed 
to  promote  economy,  efficiency,  and  effectiveness  in  the  administration  of,  and  to  prevent  and  detect 
fraud  and  abuse  in,  such  programs  and  operations.  The  Inspector  General  is  also  responsible  for  keeping 
the  Secretary  of  Defense  and  the  Congress  fully  and  currently  informed  about  problems  and  deficiencies 
relating  to  the  administration  of  such  programs  and  operations  and  the  necessity  for,  and  progress 
of,  corrective  action. 

9.4.1.8  The  Defense  Agencies.  These  agencies,  authorized  by  the  Secretary  of  Defense  pursuant  to 
the  provisions  of  Title  10,  United  States  Code,  perform  selected  support  and  service  functions  on  a 
Department  wide  basis. 

9.4. 1.9  The  OSD  Field  Activities.  These  field  activities  are  established  by  the  Secretary  of  Defense, 
under  the  provisions  of  Title  10,  United  States  Code,  to  perform  selected  support  and  service  functions 
of  a  more  limited  scope  than  the  defense  agencies. 

9.4.1.10  The  Uniformed  Services  University  of  the  Health  Sciences  (USUHS)  (DoD  Directive 
5105.45).  The  USUHS,  under  the  policy  guidance  of  the  Secretary  of  Defense  and  operational  direction 
of  a  Board  of  Regents,  is  a  fully  accredited  4-year  School  of  Medicine  whose  primary  mission  is  to 
select,  educate,  and  train  qualified  applicants  to  become  military  physicians.  The  curriculum  is  expanded 
from  that  of  the  civilian  schools  to  include  subjects  of  specific  military  importance,  such  as  command 
and  control,  tropical  medicine,  environmental  extremes,  occupational  hazards,  nonconventional  weapons, 
wartime  surgery,  and  public  health.  USUHS  also  has  educational  programs  leading  to  the  Ph.D.  degree 
in  the  basic  medical  sciences,  a  Masters  of  Public  Health  program,  and  a  continuing  medical  education 
program. 

9.4.2  Organizations  and  Functions — Office  of  the  Secretary  of  Defense 

The  Office  of  the  Secretary  of  Defense  (OSD)  is  the  principal  staff  element  used  by  the  Secretary 
of  Defense  to  exercise  direction,  authority,  and  control  over  the  Department  of  Defense.  The  mission 
of  OSD  as  an  organizational  entity,  in  coordination  with  other  elements  of  DoD,  is  as  follows: 

Develop  and  promulgate  policies  in  support  of  United  States  national  security  objectives. 

Provide  oversight  to  assure  the  effective  allocation  and  efficient  management  of  resources  consistent 
with  Secretary  of  Defense  approved  plans  and  programs. 

Develop  appropriate  evaluation  mechanisms  to  provide  effective  supervision  of  policy 
implementation  and  program  execution  at  all  DoD  levels. 

Provide  the  focal  point  for  departmental  participation  in  the  United  States  security  community 
and  other  Government  activities. 

In  addition,  each  OSD  principal  staff  official,  in  his  or  her  respective  areas  of  functional  assignment, 
is  responsible  for  performing  the  following: 

Conduct  analyses,  develop  policies,  provide  advice,  make  recommendations,  and  issue  guidance 
on  DoD  plans  and  programs. 


Develop  systems  and  standards  for  the  administration  and  management  of  approved  plans  and 
programs. 

Initiate  programs,  actions,  and  taskings  to  ensure  adherence  to  DoD  policies  and  national  security 
objectives  and  to  ensure  that  programs  are  designed  to  accommodate  operational  requirements. 

Review  and  evaluate  programs  for  carrying  out  approved  policies  and  standards. 

Inform  appropriate  organizations  and  personnel  of  new  and  significant  trends  or  initiatives  in 
assigned  areas  of  functional  responsibilities. 

Review  proposed  resource  programs,  formulate  budget  estimates,  recommend  resource  allocations, 
and  monitor  the  implementation  of  approved  programs. 

Participate  in  those  planning,  programming,  and  budgeting  activities  that  relate  to  assigned  areas 
of  functional  responsibilities. 

Review  and  evaluate  recommendations  on  requirements  and  priorities. 

Promote  coordination,  cooperation,  and  mutual  understanding  within  the  Department  of  Defense 
and  between  DoD  and  other  Federal  agencies  and  the  civilian  community. 

Serve  on  boards,  committees,  and  other  groups  pertaining  to  assigned  functional  areas,  and  represent 
the  Secretary  of  Defense  on  matters  outside  the  Department  of  Defense. 

Develop  information  and  data,  prepare  reports,  and/or  testimony  for  presentations  to  congressional 
committees  or  in  response  to  congressional  inquiries. 

Represent  the  DoD  with  congressional  committees  or  individual  Members  of  the  Congress. 
Perform  such  other  duties  as  the  Secretary  of  Defense  may  from  time  to  time  prescribe. 

9.4.2. 1  Immediate  Offices  of  the  Secretary  and  Deputy  Secretary  of  Defense.  The  Secretary  of 
Defense  is  the  principal  defense  policy  advisor  to  the  President  and  is  responsible  for  the  formulation 
of  general  defense  policy,  for  policy  related  to  all  matters  of  direct  and  primary  concern  to  the  DoD, 
and  for  the  execution  of  approved  policy.  Under  the  direction  of  the  President  and  subject  to  the 
provisions  of  the  National  Security  Act  of  1947,  as  amended,  the  Secretary  exercises  direction,  author¬ 
ity,  and  control  over  the  Department  of  Defense. 

The  Deputy  Secretary  of  Defense  assists  in  the  administration  of  the  Department.  The  Deputy 
Secretary  is  delegated  full  power  and  authority  to  act  for  the  Secretary  of  Defense  and  to  exercise  the 
powers  of  the  Secretary  upon  any  and  all  matters  concerning  which  the  Secretary  is  authorized  to  act 
pursuant  to  law. 

The  Executive  Secretary  of  the  Department  of  Defense  supports  the  Secretary  and  Deputy  Secretary 
by  executing  the  following  responsibilities:  coordinates  DoD  participation  in  the  interagency  process 
involving  national  security  management,  defense  policy,  programs  and  resources  for  the  DoD,  with 
the  White  House,  the  NSC,  State  Department,  CIA,  and  other  agencies  as  appropriate;  is  the  Secretariat 
for  both  the  Armed  Forces  Policy  Council  (AFPC)  and  the  Secretary’s  Performance  Review  (SPR) 
Board;  performs  liaison  with  the  White  House  Military  Office,  including  Presidential  support  activities; 
serves  as  the  DoD  point  of  contact  for  intergovernmental  affairs;  processes  requests  for  DoD  support 
from  the  White  House  and  other  departments/agencies;  processes  Special  Air  Mission  (SAM) 
transportation  requests  for  OSD  and  non-DoD  agencies;  manages  and  controls  all  correspondence, 
information,  and  action  documents  for  the  Secretary  and  Deputy  Secretary;  and  performs  any  special 
project  directed  by  either  the  Secretary  or  Deputy  Secretary. 

The  Assistant  to  the  Secretary  and  Deputy  Secretary  for  Executive  Personnel  is  responsible  for 
staffing  noncareer  positions  throughout  the  DoD,  approval  of  staffing  for  DoD  boards  and  committees, 


recommending  candidates  for  Presidential  boards  and  committees,  approving  appointment  of  DoD 
headquarters  level  experts  and  consultants;  acts  as  noncareer  DoD  contact  with  Office  of  Assistant 
to  the  President  for  Intergovernmental  Affairs,  and  serves  as  primary  DoD  liaison  with  the  White  House 
Personnel  Office  in  dealing  with  such  matters. 

9. 4.2. 2  Under  Secretary  of  Defense  (Policy)  (USD  (P))  (DoD  Directive  5111.1).  Under  the  direction 
of  the  Secretary  of  Defense,  the  USD(P)  is  responsible  for  the  following  functions: 

Integration  of  DoD  plans  and  policies  with  overall  national  security  objectives 

Representation  of  DoD  as  directed  in  matters  involving  the  National  Security  Council,  Department 
of  State  and  the  intelligence  community,  and  other  departments,  agencies,  and  interagency  groups 
with  responsibilities  in  the  national  security  area 

Oversight  and  coordination  of  the  formulation  and  implementation  of  DoD  planning  and  policy 
concerning  political-military  affairs,  such  as  arms  limitation  negotiations;  contingency  planning; 
intelligence  analyses  and  collection  requirements;  nuclear  weapons  and  targeting;  communications, 
command,  and  control  (C3);  and,  the  use  of  outer  space 

Oversight  and  coordination  of  policy  review  concerning  intelligence  planning  and  requirements, 
counterintelligence  and  investigative  programs,  security  plans  and  programs,  and  sensitive  intelligence 
matters  including  arrangements  with  foreign  governments 

Review  of  evaluations  and  the  development  of  recommendations  to  the  Secretary  of  Defense 
concerning  plans  and  requirements  for,  and  capabilities  of,  existing  or  proposed  United  States 
or  foreign  forces  and  their  deployment  with  particular  attention  to  performance  of  missions  which 
are  or  may  be  critical  in  consideration  of  United  States  national  security  policy 

Coordination  of  DoD  participation  in  the  preparation  of,  and  followthrough  on,  NATO  Short- 
Term  Initiatives  and  Long-Term  Programs,  and  the  integration  of  NATO  considerations  in  the 
development  and  formulation  of  DoD  decisions 

Law  of  the  Sea 

Military  Assistance  Advisory  Groups  (MAAG)  and  missions  pertaining  to  security  assistance. 
Negotiation  and  monitoring  of  agreements  with  foreign  governments 

Development  of  DoD  policy  positions,  recommendations,  and  coordination  of  all  matters  concerning 
disarmament  and  arms  control  to  include  START  and  MBFR  and  other  Defense-related  international 
negotiations 

DoD  focal  point  for  long  and  midrange  policy  planning  on  strategic  international  security  matters 

Formulation  of  policy  related  to  strategic  offensive  and  defensive  forces,  theater  nuclear  matters 
and  capabilities,  arms  control  negotiations,  and  the  relationship  between  strategic  and  theater  force 
planning  and  budgets 

Oversight  of  DoD  activities  related  to  the  North  Atlantic  Treaty  Organization  and  East-West 
economic  policy,  including  East-West  trade,  technology  transfer  issues,  and  issues  affecting  the 
defense  industrial  mobilization  base 

Formulating,  planning,  conducting,  and  preparing  net  assessments  for  the  Secretary  of  Defense 

Formulating  plans  and  policy  related  to  general  purpose  forces,  non-European  regional  security 
requirements,  and  related  budget  considerations 

Developing  policies,  plans,  and  procedures  for  the  discharge  of  Department  of  Defense  functions 
in  national  emergencies;  providing  support  to  DoD  and  other  U.S.  Government  or  State  agencies 
on  civil  defense  and  related  matters. 
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The  above  functions  are  carried  out  through  the  following  key  personnel: 

Deputy  Under  Secretary  of  Defense  for  Policy 

Assistant  Secretary  of  Defense  (International  Security  Affairs) 

Assistant  Secretary  of  Defense  (International  Security  Policy) 

Director  of  Net  Assessment. 

In  addition,  the  Under  Secretary  of  Defense  for  Policy  exercises  direction,  authority,  and  control 
over  the: 

Defense  Investigative  Service 
Defense  Security  Assistance  Agency. 

9.4. 2.3  Under  Secretary  of  Defense  (Research  and  Engineering)  (USDRE)  (DoD  Directive 
5129.1).  Under  the  direction  of  the  Secretary  of  Defense,  the  USDRE  is  responsible  for  the  following 
functions: 

Scientific  and  technical  information 
Basic  and  applied  research 

Design  and  engineering,  including  life-cycle  considerations 

Development  and  acquisition  of  weapon  systems,  including  procurement  policy  and  production 
planning.  This  function  includes  national,  strategic,  and  tactical  communications;  command  and 
control;  and  intelligence  activities. 

Development  test  and  evaluation  in  accordance  with  DoD  Directive  5000.3,  to  include  review  and 
approval  of  the  T&E  Master  Plan  (TEMP) 

Environmental  services 

Assignment  and  reassignment  of  research,  engineering  and  acquisition  responsibility  for  systems, 
activities,  and  programs 

Coproduction  and  research  interchange  with  friendly  and  allied  nations,  in  conjunction  with  the 
Under  Secretary  of  Defense  for  Policy 

Contract  placement  and  administration  for  research,  development,  and  weapon  systems  acquisition 
programs 

Military  applications  of  atomic  energy,  nuclear  weapons  development  and  acquisition,  security, 
safety,  R&D,  deployment,  employment  and  targeting,  and  theater  nuclear  force  modernization. 

The  above  functions  are  carried  out  with  the  support  of  the  following  key  personnel: 

Assistant  Secretary  of  Defense  (Command,  Control,  Communications,  and  Intelligence) 
Assistant  Secretary  of  Defense  (Development  and  Support) 

Assistant  Secretary  of  Defense  (Research  and  Technology) 

Assistant  to  the  Secretary  of  Defense  (Atomic  Energy) 

Deputy  Under  Secretary  (Acquisition  Management) 

Deputy  Under  Secretary  (International  Programs  and  Technology) 

Deputy  Under  Secretary  (Research  and  Advanced  Technology) 

Deputy  Under  Secretary  (Strategic  and  Theater  Nuclear  Forces) 


Deputy  Under  Secretary  (Tactical  Warfare  Programs) 

Director,  Test  and  Evaluation. 

In  addition,  the  USDRE  exercises  direction,  authority,  and  control  over  the  following: 

Defense  Advanced  Research  Projects  Agency 
Defense  Communications  Agency 
Defense  Mapping  Agency 
Defense  Nuclear  Agency. 

9.4.2.4  Assistant  Secretary  of  Defense  (Comptroller)  (ASD(C»  (DoD  Directive  5118.3).  Under  the 
direction  of  the  Secretary  of  Defense,  the  ASD(C)  is  responsible  for  the  following  functions: 

Planning,  programming,  and  budgeting  system  (PPBS),  including  programming  coordination  and 
control 

DoD  budget  formulation  and  execution,  resources  allocation,  and  surveillance  over  utilization 
Focal  point  for  budgeted  savings  under  economy  and  efficiency  initiatives 
Information  to  support  justification  of  the  budget  to  Congress 

Focal  point  for  joint  OSD  and  Office  of  Management  and  Budget  (OMB)  review  of  the  budget 

Coordination  with  OMB  on  management  reviews  and  analyses  performed  in  connection  with  the 
budget  process 

Initiatives  to  strengthen  Departmentwide  resource  management  and  to  improve  management 
information  provided  to  senior  officials 

Financial  management  including  financial  accounting  and  reporting  systems;  internal  control 
systems;  pricing  policy  including  Foreign  Military  Sales  pricing;  banking  and  credit  union  services 
on  DoD  installations;  and  international  financial  affairs 

Policies  and  procedures  on  the  reporting,  preparation,  and  dissemination  of  statistical  information 

Senior  DoD  official  for  information  resources  management  including  oversight  of  acquisition  and 
use  of  information  technology  and  related  resources  for  business  and  administrative  purposes 

Organizational  analysis  and  management  planning 

DoD  privacy  program  in  compliance  with  the  Privacy  Act  of  1974 

OSD  historical  program  and  DoD  historical  program  coordination 

Policy  guidance  and  coordination  on  matters  of  administrative  support  received  or  provided  by 
DoD  components 

Cost  performance  measurement  systems 

nnim  fnr  selected  acauisition  reports  (SARs)  and  unit  cost  reporting 
stuc^es  analyses  related  to  comptroller  responsibilities 

Member  and  Executive  Secretary  of  the  Defense  Resources  Board,  member  of  the  Cost  Analysis 
Improvement  Group,  member  of  the  Defense  Systems  Acquisition  Review  Council,  member  and 
Executive  Secretary  of  the  DoD  Council  on  Integrity  and  Management  Improvement,  and  Chairman 
of  the  Major  Automated  Information  Systems  Review  Council. 

In  addition,  the  ASD(C)  exercises  direction,  authority,  and  control  over  the  following: 

Defense  Contract  Audit  Agency 


Washington  Headquarters  Services  (through  the  Deputy  Assistant  Secretary  of  Defense  (Admini¬ 
stration)  who  has  collateral  responsibility  as  Director,  Washington  Headquarters  Services). 

9.4.2.5  Assistant  Secretary  of  Defense  (Manpower,  Installations  and  Logistics)  (ASD(MIAL))  (DoD 
Directive  5124.1).  Under  the  direction  of  the  Secretary  of  Defense,  the  ASD(MIAL)  is  responsible 
for  the  following  functions: 

Total  force  structure  analysis  as  related  to  quantitative  and  qualitative  manpower  requirements, 
manpower  utilization,  logistics,  readiness,  and  support 

The  allocation  of  the  total  force  structure  among  DoD  components  and  between  the  active  and 
reserve  components  within  the  military  departments 

Civilian  and  military  personnel  management  programs  and  systems,  including  attraction  and 
retention  of  military  personnel;  personnel  utilization;  compensation,  retired  pay,  per  diem,  travel, 
and  transportation  allowances;  civilian  and  military  personnel  career  development,  training,  and 
education;  labor-management  relations;  morale,  discipline,  and  welfare;  and  community  services 

Development  of  civilian  and  military  manpower  programs  and  logistics  programs  to  meet  peacetime 
readiness  and  wartime  sustainability  requirements  of  the  DoD 

Nonappropriated  fund  instrumentalities 

Weapons  support 

Logistics 

Equal  opportunity,  equal  employment  opportunity,  and  DoD  contractor  compliance  with  equal 
employment  opportunity  requirements  in  government  contracts 

Readiness 

Energy 

Installations  management 
Conservation  of  resources 
Economic  adjustment 

Review  and  evaluation  of  the  requirements  of  the  Defense  System  Acquisition  Review  Council 
(DSARC)  weapon  programs  and  proposed  weapon  systems  for  adequacy  of  readiness  goals  and 
resources,  including  manpower,  personnel,  training,  logistics,  installations  support,  reliability, 
maintainability,  and  design  safety. 

In  addition,  the  ASD(MIAL)  exercises  direction,  authority,  and  control  over  the  following: 

Armed  Forces  Chaplains  Board 

Defense  Logistics  Agency 

Department  of  Defense  Dependents  Schools 

Office  of  Ecnonomic  Adjustment 

DoD  Explosives  Safety  Board 

Defense  Race  Relations  Institute 

Defense  Advisory  Committee  on  Women  in  the  Services. 

9.4.2.6  Director  of  Operational  Test  and  Evaluation  (DOT&E)  (Title  10,  United  States  Code,  Section 
136a)  (DoD  Directive  5141.2).  Under  the  provisions  of  Title  10,  U.S.C.  136a,  and  under  the  direction 
of  the  Secretary  of  Defense,  the  DOT&E  is  the  principal  staff  assistant  and  advisor  to  the  Secretary 


of  Defense  on  OT&E  in  the  DoD  and  the  principal  OT&E  official  within  the  senior  management  of 
the  DoD.  In  this  capacity,  the  DOT&E  is  responsible  for  the  following  functions: 

Prescribe  policies  and  procedures  for  the  conduct  of  OT&E  within  the  Department  of  Defense. 

Provide  advice  and  make  recommendations  to  the  Secretary  of  Defense,  and  issue  guidance  to 
and  consult  with  the  heads  of  the  DoD  components  with  respect  to  OT&E  in  the  DoD  in  general, 
and  with  respect  to  specific  OT&E  to  be  conducted  in  connection  with  a  major  defense  acquisition 
program. 

Designate  selected  special  interest  weapons,  equipment,  or  munitions  as  major  defense  acquisition 
programs. 

Develop  systems  and  standards  for  the  administration  and  management  of  approved  OT&E  plans 
for  major  defense  acquisition  programs. 

Monitor  and  review  all  OT&E  in  the  DoD  to  ensure  adherence  to  approved  policies  and  standards. 
Coordinate  operational  testing  conducted  jointly  by  more  than  one  DoD  component. 

Review  and  make  recommendations  to  the  Secretary  of  Defense  on  all  budgetary  and  financial 
matters  relating  to  OT&E,  including  operational  test  facilities  and  equipment. 

Initiate  plans,  programs,  actions,  and  taskings  to  ensure  that  OT&E  for  major  defense  acquisition 
programs  is  designed  to  evaluate  the  operational  effectiveness  and  suitability  of  U.S.  military  weapon 
systems. 

Review  and  report  to  the  Secretary  of  Defense  on  the  adequacy  of  operational  test  planning, 
priorities,  support  resources,  execution,  evaluation,  and  reporting  for  major  defense  acquisition 
programs  while  avoiding  unnecessary  duplication. 

9.4.2.7  Director,  Program  Analysis  and  Evaluation  (DPA&E)  (DoD  Directive  5141.1).  Under  the 
direction  of  the  Secretary  of  Defense,  the  DPA&E  is  responsible  for  performing  analyses,  identifying 
issues,  and  evaluating  alternative  programs  for  the  following  functions: 

Force  review  of  active  and  reserve  components 
Strategic  and  theater  nuclear  forces 

Weapon  systems  and  major  items  of  material,  including  critical  reviews  of  requirements, 
performance,  and  life-cycle  costs  of  current  and  proposed  weapon  systems 

Nuclear  warhead  requirements 
Support  systems 

Deployment  plans  and  overseas  basing  requirements 
Mobility  force  programs  and  prepositioning  plans 
Material  support  programs  and  war  reserve  stocks 
Force  readiness  and  capabilities 

Implications  for  manpower  resources  of  specific  force  structure  plans 

Contingency  plans 

Security  assistance  programs 

Allied  and  foreign  military  requirements  and  capabilities. 
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DNSARC/DON  RESOURCES  BOARD*  . 

Membership: 

•  ASN  (R,E£rS)  •  CNO 

•  ASN  (M&RA)  •  CMC 

•  ASN  (SB&L)  •  CNM 

•  DEPUNDERSECNAV  (FM) 

•  Director,  OPA  (Executive  Secretary) 

•  Director,  Office  of  the  General  Counsel 

*  As  DON  Resources  Board,  chaired  by 
UNDERSECNAV,  insure  proper  correlation 
between  the  acquisition  process  and  PPBS  cycle, 
advising  SECNAV  on  annual  POM  and  budget 
submission  to  OSD. 


1M.19UMQ 


KEEP  HIM/HER  INFORMED 


VY  MISSION  NEED  REVIEW  AND  APPROVAL 


NAVY  PROGRAM  INITIATION 
PROCEDURES  (OPNAVINST  5000 


NAVY  MILESTONE  III  DECISIONS 


•  Approved  for  Full  Production  (AFP) 

•  System  meets  all  technical  and  operational 
thresholds 

•  ILS  requirements  fully  demonstrated  during 
OTErE 

•  No  additional  development  or  corrective  action 
required 

•  Approved  for  Limited  Production  (ALP) 

•  Aim  at  AFP  for  following  year  AFP/ ALP  decision 

•  No  more  than  1  year's  production 

•  COMOPTEVFOR  considers  system  operationally 
effective  and  suitable,  with  clear  plan  and  funding 
for  corrections 

•  Not  approved  for  production 
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SECTION  10 

TECHNICAL  INFORMATION  SUPPORT 
H.  Blanchard,  Code  96 


10.1  INTRODUCTION 

10.1.1  References 

DoD  Instruction  5200.21;  Dissemination  of  DoD  Technical  Information,  27  September  1979 

DoD  Directive  3200.12,  DoD  Scientific  and  Technical  Information  Program,  15  February  1983 

SECNAV  Instruction  3900.43,  Navy  Scientific  and  Technical  Information  Program,  29 
December  1983 

NAVMAT  Instruction  4160.2,  Technical  Manual  Management,  24  November  1980 

NAVSEA  Instruction  4160.3,  Technical  Manual  Management  Program  (TMMP),  August  1982 

NOSC  Instruction  4160. 1A,  Technical  Manual  Procedures,  1  November  1985 

NOSC  Instruction  5600. 2D 

NOSC  Instruction  5400. 2D 

NOSC  TD  178,  Revision  B 

NOSC  TD  611 

NOSC  TD  841,  Distribution  Statements  for  Technical  Publications 
NOSCINST  3150.1,  Photographic  Operations  and  Material  Control 
NOSCINST  3150.2,  Photographic  Operations  and  Material  Control 
NOSCINST  3150.3,  AV  Equipment  Pool 
NOSCINST  3158. 1A,  Presentation  Aids 

NOSCINST  5290.1,  Photographic,  Video  and  Audiovisual  Operations  and  Material  Control 
(will  supersede  the  above  3150  series) 

OPNAVINST  5290.1,  Navy  Audiovisual  Management  and  Operations  Manual 
NOSC  TD.220,  A  Guide  for  NOSC  Presentation  Graphics 
NOSCINST  5070. IB,  NOSC  Library  Services,  19  November  1985 

NOSCINST  3900. 2B,  Initiation  of  New  Technical  Work  Assignments;  Policies  and  Procedures 
for  6  April  1984 

NOSCINST  4200.5,  Procurement  Requirements  Package  (PRP)  Handbook,  30  August  1978 

Effective  Business  and  Technical  Presentations ,  by  George  L.  Morrises 

Hon  to  Prepare.  Stage,  and  Deliver  Winning  Presentations ,  by  Thomas  Leech 
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10.1.2  Outline 


Introduction 

References 

Outline 

Summary 

Publication  Requirements 
Documentation  Requirements 
Research  and  Engineering 
Technical  Manuals 
Distribution  Requirements 
Distribution  Statements 
Research  and  Engineering 
Technical  Manuals 
Planning  for  Publications 
Research  and  Engineering 
Technical  Manuals 
Contract  Requirements 
Research  and  Engineering 
Technical  Manuals 
Audiovisual  Support 
Visual  Media  Definitions 
Visual  Documentation 
Visual  Media  Products  and  Services 
Guidelines  for  Viewgraphs 
Planning  Considerations  for  Visual  Media 
Contacts  for  Assistance 
Technical  Library  Services 
Library  Collections 
Acquisitions  Policy 
Library  Access 

Reference/Literature  Search  Services 
Library  Publications 
Special  Projects  and  Services 
Effective  Presentations 

10.1.3  Summary 

Project  documentation  includes  requirements  for  the  publication  of  RDT&E  results  (in-house  and 
contract)  and  technical  manuals.  Various  DoD,  Navy,  and  NOSC  instructions  specify  requirements 
and  procedures  for  preparation,  production,  and  distribution  of  those  documents.  Lack  of  proper 
documentation  or  poorly  prepared  documentation  adversely  affects  both  DoD’s  RDT&E  program  and 
the  life  cycle  of  equipment  or  systems.  Project  managers  are  responsible  for  consulting  Code  961  at 
the  beginning  of  their  projects  to  ensure  proper  planning  and  adequate  funding  to  meet  the  specific 
requirements. 

Effective  visual  communcations  are  key  elements  in  project  management.  Photographic  and  video 
documentation  and  illustrations  of  experiments,  concepts  tests,  analysis  of  events,  and  ongoing  historical 
recordings  are  highly  visible  and  tangible  products  from  RDT&E  efforts.  A  visual  report  produced 


from  graphic  and  video  media  can  ensure  successful  competition  for  funds  and  the  imparting  of 
knowledge  and  information  in  the  most  direct  way.  Further,  the  costs  of  obtaining  the  visual  record 
are  more  than  offset  by  savings  of  valuable  scientific  and  engineering  manhours  that  would  otherwide 
be  devoted  to  reports  and  briefs. 

NOSC  visual  media  specialists  are  dedicated  to  the  needs  of  the  RDT&E  programs  and  projects. 
As  part  of  the  RDT&E  team,  they  possess  the  unique  insight  to  capture  the  visual  record  in  the  most 
cost  effective  way  for  your  project. 

To  maximize  the  value  of  visual  documentation,  the  most  important  consideration  is  to  include 
visual  media  as  a  data  requirement  early  into  the  program.  This  will  allow  for  funding  considerations 
of  these  requirements. 

The  Technical  Libraries  Branch  offers  a  full  range  of  library  services  as  detailed  in  NOSCINST 
5070. 1 B.  Certain  aspects  of  these  services  are  particularly  important  to  the  project  manager  in  planning 
for  meeting  project  technical  information  needs. 

Finally  a  few  words  on  the  importance  of  presentations  complete  the  section. 


10.2  PUBLICATION  REQUIREMENTS 


10.2.1  Documentation  Requirements 


v  10.2.1.1  Research  and  Engineering.  DoD’s  Scientific  Technical  Information  Program  (STIP)  requires 

that  all  significant  results  derived  from  DoD  endeavors,  including  those  generated  under  contract,  be 
recorded  as  technical  documents.  These  documents  are  to  be  made  available  to  the  DoD  research  and 
engineering  community  through  supporting  technical  libraries  and  the  Defense  Technical  Information 
Center  (DTIC).  STIP  also  requires  that  these  documents  be  prepared  and  distributed  without  undue 
delay  and  according  to  established  standards  for  format,  security  markings,  and  reproducibility. 

NOSC  instructions  reiterate  these  requirements  and  establish  the  following  procedures  for  their 
implementation: 

All  NOSC  publications  intended  for  external  distribution  must  carry  a  NOSC  publication  number 
assigned  by  the  Publications  Branch.  (The  NOSC  publication  series  includes  technical  reports, 
technical  documents,  technical  notes,  contractor  reports,  and  technical  manuals.) 

The  routing  cycle  for  these  publications  includes  branch  and  division  heads,  security,  and,  if 
unclassified,  public  affairs. 

Primary  and  secondary  distribution  of  these  documents  is  the  responsibility  of  Code  961. 

Code  961  has  sole  authority  at  NOSC  for  sending  publications  to  DTIC. 

These  procedures  and  a  discussion  of  the  Center’s  different  publication  series  can  be  found  in 
NOSC  TD  178. 

10.2.1.2  Technical  Manuals.  Requirements  concerning  technical  manuals  are  based  on  the  DoD 
concept  that  they  are  a  part  of  the  integrated  logistic  support  (1LS)  process.  They  contain  the  logistic 
data  necessary  to  install,  operate,  maintain,  repair,  and  otherwise  support  Navy  equipment  and  systems. 
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They  are  required  when  systems  or  equipment  are  being  built  or  equipment  is  integrated  into  a  system, 
and  they  must  be  prepared  according  to  specifications  that  control  their  content  and  format. 

In  addition  to  reiterating  these  requirements,  NOSC  instructions  also  specify  the  following: 

Technical  manuals  developed  for  NOSC  systems  and  equipment  must  be  accurate,  comprehensible, 
economically  produced,  and  available  to  meet  user  requirements  throughout  the  life  cycle  of  the 
equipment. 

The  cost  of  technical  manuals  must  be  considered  as  part  of  the  overall  cost  for  delivery  of  an 
operational  system  or  piece  of  equipment. 

Technical  manuals  must  be  included  either  as  a  contract  line  item  number  and  contract  exhibit 
or  as  a  data  item  description  in  the  acquisition  phase. 

Technical  manual  specifications  must  be  tailored  in  the  contractual  requirements  to  provide  for 
factors  such  as  user  profile,  maintenance  plan,  and  training  requirements. 

All  technical  manual  efforts  must  be  coordinated  through  the  NOSC  Publications  Branch.  Included 
are  such  items  as  logistics  planning,  in-process  reviews,  acquisitions,  validation,  verification,  and 
changes. 

In  addition,  NOSC  instructions  specify  that  Code  961  has  sole  authority  at  the  Center  to  assign 
technical  manual  identification  numbers,  including  those  required  by  system  commands.  These  numbers 
are  assigned  only  if  the  NOSC  and  DoD  requirements  for  technical  manuals  are  met. 

Requirements  of  the  sponsoring  systems  command  must  also  be  met.  For  example,  NAVSEA 
requires  the  following: 

An  approved  Technical  Manual  Contract  Requirements  (TMCR)  or  Technical  Manual  SEATASK 
Requirements  (TMSR)  must  be  used  as  the  specifications-tailoring  document.  Without  this  TMCR 
or  TMSR,  NAVSEA  will  not  issue  a  technical  manual  identification  number. 

A  Technical  Manual  Quality  Assurance  (TMQA)  Program,  which  includes  validation  and 
verification,  must  be  budgeted  for  and  implemented  in  all  technical  manual  procurements. 


10.2.2  Distribution  Requirements 

10.2.2.1  Distribution  Statements.  NOSC  requires  that  all  publications  with  a  NOSC  publication 
number  carry  a  formal  distribution  statement.  The  purpose  of  this  statement  is  to  control  the  secondary 
distribution  of  the  publication.  Distribution  statements  are  assigned  by  security  and  public  affairs  after 
the  publication  has  been  reviewed  for  technical  adequacy  by  branch  and  division  heads.  NOSC  forms 
5605  and  5720  are  used  for  this  purpose. 

10.2.2.2  Research  and  Engineering.  Research  and  engineering  publications— whether  research, 
software,  or  engineering— must  be  provided  to  DTIC.  If  the  information  is  so  sensitive  that  a  copy 
of  the  publication  cannot  be  given  to  DTIC,  DTIC  must  be  provided  with  a  bibliographic  citation. 
At  NOSC,  Code  961  does  this  by  means  of  the  DD  Form  1473.  Information  up  to  and  including  secret 
is  sent  to  DTIC. 

Those  publications  which  are  specifically  related  to  subjects  such  as  chemical  propulsion,  tactical 
weapons  guidance  and  control,  infrared  technology,  plastics,  reliability,  and  shock  and  vibration  are 
also  sent  to  appropriate  information  analysis  centers.  (These  centers  are  administratively  managed  and 
funded  by  the  Defense  Logistics  Agency  and  DTIC.) 


10-6 


1 


I 


.1 

1 


I 


10.2.2.3  Technical  Manuals.  NOSC-numbered  technical  manuals  can  be  provided  to  sponsors,  but 
cannot  be  sent  to  the  Navy  Publications  and  Form  Center  (NPFC)  or  used  on  shipboard  except  during 
the  experimental  phase  of  the  project. 

Those  manuals  which  carry  system  command  numbers  are  sent  to  NPFC  for  distribution  to  Fleet 
users.  Distribution  is  controlled  by  the  distribution  statement  that  appears  on  the  manual. 

Technical  manuals  are  not  sent  to  DTIC  or  information  analysis  centers. 

10.2.3  Planning  for  Publications 

10.2.3.1  Research  and  Engineering.  Planning  for  this  type  of  publication  must  begin  at  the  conceptual 
phase  of  a  project  and  continue  throughout  its  entire  life  cycle.  Information  to  be  reported  includes 
the  following: 

a.  Research.  Information  that  contributes  to  increased  knowledge  of  natural  phenomena  and  the 
environment;  to  the  solution  of  problems  in  the  physical,  behavioral,  social,  and  management 
sciences;  or  to  the  expansion  of  knowledge  in  scientific  areas. 

b.  Development.  Information  that  involves  the  extension  of  theoretical,  practical,  and  useful 
application  of  basic  designs,  ideas,  and  scientific  concepts  or  pieces  of  equipment.  The  dominant 
characteristic  of  these  efforts  is  that  they  are  pointed  to  specific  military  problem  areas  with 
a  view  to  developing  feasibility  and  practicality  of  proposed  solutions  and  determining  usage, 
applications,  and  effectiveness  parameters. 

c.  Test.  Information  that  documents  the  procedures  and  results  of  subjecting  items,  systems, 
materials,  personnel,  or  techniques  to  simulated  or  actual  operational  conditions  to  determine 
characteristics,  suitability,  and  compliance  with  specific  requirements. 

d.  Evaluation.  Information  that  provides  values,  appraisals,  or  results  relevant  to  strengths, 
weaknesses,  feasibility,  potential,  and  military  worth  of  efforts,  concepts,  or  hardware. 

The  important  thing  to  remember  is  that  the  information  must  be  pertinent  and  timely  to  subjects 
designated  in  DoD  budget  reports.  Relevancy  of  subject  contents  to  ongoing  RDT&E  activities  is  essential. 

Approximately  5  percent  of  project  funding  should  be  allocated  to  prepare  these  publications. 

10.2.3.2  Technical  Manuals.  Planning  for  technical  manuals  must  also  occur  at  the  beginning  of 
the  project.  Development  of  technical  manuals  for  new  systems  or  equipment  must  be  initiated  either 
concurrently  with  performance  requirements  or  at  the  earliest  possible  time  in  the  conceptual  phase. 
Technical  manuals  for  commercially  available  equipment  must  be  developed  during  the  procurement 
phase. 

The  cost  of  technical  manuals,  depending  upon  the  type  required  and  their  intended  purpose,  ranges 
from  $300  to  $1,000  per  page.  Specific  costs  for  a  project  can  be  obtained  when  Code  961  reviews 
the  project’s  requirements. 

The  time  required  for  logistics  certification,  as  well  as  the  time  required  fo:  writing  and  production, 
must  be  included  during  the  program  and  acquisition  phases. 

Plans  must  also  be  made  to  tailor  the  applicable  specifications  to  meet  system  requirements.  This 
tailoring  will  vary  for  each  project  and  will  depend  upon  factor';  such  as  system  command  requirements, 
users,  and  planned  maintenance. 


10.2.4  Contract  Requirements 


10.2.4.1  Research  and  Engineering.  The  contractor  must  provide  publications  that  are  suitable  for 
entry  into  DTIC.  This  means  that  the  information  must  be  complete,  legible,  reproducible,  technically 
accurate,  and  sufficiently  detailed  to  permit  use  of  the  publication. 

The  best  data  item  descriptions  (DIDs)  to  use  for  ordering  your  publications  are  those  that  specify 
the  military  standard  or  specification  that  applies  to  the  subject  matter  being  reported.  For  example, 
UDI-A-26199A  and  DI-S-4057  are  excellent  for  technical  reports. 

All  publications  ordered  should  be  subject  to  a  preliminary  government  review  before  final  delivery 
of  the  data  deliverable.  During  this  review,  subject  matter  experts  outside  the  project  should  be  asked 
to  review  the  data.  For  example.  Code  961  can  review  research  reports  or  final  reports  before  the  final 
acceptance  is  made.  This  allows  any  problems  to  be  identified  and  then  rectified  by  the  contractor. 

10.2.4.2  Technical  Manuals.  The  ordering  of  technical  manuals  can  involve  outlines,  manuscripts, 
preliminary  manuals,  and  camera-ready  copy.  The  items  ordered  depend  upon  the  specific  project. 

Current  policy  dictates  that  technical  manuals  be  ordered  as  data  on  the  DD  form  1423  (CDRL). 
Typical  data  item  descriptions  (DIDs)  are  DI-M-2041,  Technical  Manual  Outline/Bookplan;  DI-M-2042, 
Technical  Manual  Manuscript  Copy;  DI-M-2043A,  Manual,  Technical,  Preliminary;  DI-M-2044A, 
Manual,  Technical,  Standard;  and  DI-M-2046,  Technical  Manual  Permanent  Change  Pages. 

In  the  near  future,  the  method  for  ordering  technical  manuals  will  change  and  the  CDRL  will 
no  longer  be  used  for  this  purpose.  Requirements  for  manuals  are  to  be  contract  line  items  and  contract 
exhibits,  the  exact  forms  of  which  are  yet  to  be  determined. 

Supporting  technical  manual  data  items  are  and  will  be  ordered  via  the  CDRL.  Possible  data  items 
include:  DI-M-2040A,  Technical  Manual  Organization  Plan;  DI-M-2194,  Technical  Manual  Quality 
Assurance  Program  Plan;  DI-M-2195,  Validation  Plan;  and  DI-M-2196,  Validation  Certification. 

For  NAVSEA  requirements,  an  approved  TMCR  or  TMSR  is  mandatory  for  use  in  all  procurements 
of  (NAVSEA)  technical  manuals,  permanent  changes,  or  revisions. 


10.3  Audiovisual  Support 

The  intent  of  this  subsection  is  to  outline  the  visual  media  services  available  and  how  you  may 
obtain  them  for  your  project  support. 


10.3.1  Visual  Media  Definitions 

The  following  definitions  are  extracted  from  OPNAVINST  5290.1,  Navy  Audiovisual  Management 
and  Operations  Manual,  and  apply  to  all  local  instructions  as  well. 

10.3.1.1  Official  Navy  Photography.  This  includes  those  functions  that  are  concerned  with  aerial, 
surface,  and  underwater  still,  motion  picture,  audio,  and  video  documentation  necessary  to  support 
the  Naval  establishment.  For  this  instruction,  “Official  Navy  Photography”  includes  still,  motion  picture, 
photographic  instrumentation,  and  audio  and  video  documentation,  along  with  the  film,  prints,  and 
tapes  produced  at  government  expense  by  government  employees  or  by  contract  for  the  government. 


10.3.1.2  Technical  Documentation.  This  includes  audiovisual  resources  dedicated  to  documentation 
of  technical  subjects  for  the  purpose  of  supporting  the  RDT&E  mission.  It  also  includes  resources  defined 
above  as  “Official  Navy  Photography”  as  well  as  film  and  video  reports  of  a  basic  nature  (recorded 
narration  and  approved  titles),  slide  and  audio  presentations,  graphic  arts  training  aids  and  devices, 
and  audiovisual  design  subfunctions  where  the  purpose  is  to  record,  propose,  report,  or  convey  technical 
data  on  RDT&E  programs. 

10.3.1.3  Audiovisual  (AV)/Visual  Media.  This  involves  the  use  of  sound  or  visual  imagery  or  both 
to  communicate  information.  It  includes  the  use  of  motion  pictures,  video,  still  photographs,  slides 
and  film  strips,  audio,  graphic  illustrations,  models,  and  displays.  (Visual  media  is  basically  the  same 
as  audiovisual.) 

10.3.1.4  Audiovisual  Activity.  This  is  an  organizational  element  or  a  function  within  an  organization 
in  which  one  or  more  individuals  are  classified  as  audiovisual  or  whose  principal  activity  is  to  provide 
AV  services  and  products  or  to  manage  AV  resources.  The  term  applies  but  is  not  limited  to  AV 
equipment,  facilities,  products,  personnel,  maintenance,  supplies,  procurement,  and  budget.  AV  activities 
include  those  that  expose  and  process  original  photography;  record,  distribute,  or  broadcast  electronically 
(video  and  audio);  reproduce  still  and  motion  picture  photography;  duplicate  electronic  recordings; 
produce  AV  products;  provide  AV  services;  distribute  or  preserve  AV  products;  prepare  graphic  artwork; 
fabricate  AV  training  aids,  models,  and  displays;  provide  presentation  services;  or  manage  any  of  these 
activities. 

10.3.1.5  Audiovisual  Product.  This  includes  audiovisual  media  elements  such  as  still  photography, 
graphic  arts,  still  projections  (overhead  transparencies,  slides,  and  film  strips),  motion  pictures  (film 
videotape  and  videodisc),  and  audio  recordings  (tape  and  disc).  Production  is  a  unique  form  of  AV 
product  and  is  usually  addressed  separately. 

10.3.1.6  Audiovisual  Production.  An  AV  production  is  a  unified  presentation  which  contains  sound 
or  visual  imagery  or  both;  is  titled,  edited,  and/or  accompanied  by  sound;  and  conveys  a  message  through 
a  recorded  medium  or  broadcast.  The  term  may  also  apply  to  combining  or  arranging  any  separate 
or  combined  audio  or  visual  product(s)  in  continuity  according  to  a  plan  or  script.  A  production  is 
the  end-item  of  the  production  process.  This  term  is  synonymous  with  the  Office  of  Management  and 
Budget  (OMB)  use  of  the  term  “audiovisual  product.” 

10.3.1.7  Optical  Instrumentation.  This  involves  the  use  of  optical  systems,  coupled  with  photographs 
or  television  recording  devices  which  may  include  audio,  to  record  scientific  and  engineering  phenomena 
for  measurement  and  analysis.  It  may  include  the  recording  of  data  to  correlate  optical  images  to  time, 
space  positions,  or  other  engineering  data. 

10.3.1.8  Official  Photographer.  This  is  an  individual  whose  sole  employment  is  making  official 
photographs,  motion  pictures,  video  recordings,  or  production  of  RDT&E  products  and  audiovisual 
presentations  developed  from  these  products.  An  official  Photographer’s  Pass  is  issued  only  to  such 
individuals. 

10.3.1.9  Authorized  Photographer/Recorder.  This  is  an  employee  who  occasionally  needs  to  use 
a  camera,  video,  or  sound  recorder.  The  Camera/Video/Recording  Equipment  Pass  (form  NOSC-SD 
5512/21)  is  issued  to  these  persons  for  a  period  not  to  exceed  12  months. 
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10.3.1.10  Audiovisual  Equipment.  The  AV  equipment  can  fall  into  two  categories: 

a.  Items  of  a  durable  nature  that  are  capable  of  continuing  or  repetitive  use  by  an  individual  or 
organization  for  recording,  production,  reproduction,  processing,  and  exhibiting  AV  products 
or  documentation.  (Included  are  photographic,  television,  videotape  or  videodisc,  audiotape 
or  audiodisc,  graphic  arts,  and  computer  graphics  equipment.) 

b.  Items  that  have  an  AV  function  as  an  integral  part  of  a  non-AV  system  or  device  (existing 
or  under  development)  and  that,  when  permanently  removed,  can  be  identified  as  an  end-item 
of  equipment. 

10.3.1.11  Dedicated  AV  Support  Activity  (DAVSA).  This  is  an  AV  activity  that  provides  dedicated 
audiovisual  support  which  is  integral  to  the  performance  of  the  primary  mission(s)  of  the  Center.  It 
does  not  include  productions,  production  services,  and  related  support  functions. 

10.3.1.12  Graphic  Arts.  This  relates  to  the  design,  creation,  and  preparation  of  two-  and  three- 
dimensional  visual  aid  products.  It  includes  charts;  graphs,  posters;  visual  materials  for  television,  motion 
pictures,  and  publications;  displays  and  presentations;  and  exhibits  prepared  manually,  by  machine, 
or  by  computer. 

10.3.1.13  Computer-Generated  Graphics  System.  This  is  an  integrated  computer,  minicomputer, 
or  microcomputer  and  software  system  designed  and  intended  primarily  for  generation  of  graphic  arts 
productions;  or  a  system  composed  of  selected  computer,  minicomputer,  or  microcomputer  hardware 
components,  plotters,  and  software  systems  whose  primary  purpose  is  to  produce  graphic  arts  displays, 
charts,  and  pictures. 

10.3.1.14  Major  Claimant.  This  is  an  organization  directly  subordinate  to,  established  by  authority 
of,  and  specifically  designated  by  the  Secretary  of  the  Navy.  (For  NOSC,  it  is  the  Office  of  Space  and 
Naval  Warfare  Systems  Command,  Code  512.) 

10.3.1.15  Technical  Report.  This  is  an  AV  report,  an  assemblage  of  film  or  video  clips,  or  an  assembly 
of  technical  documentation  to  report  on  a  single,  mission-related  event. 


10.3.2  Visual  Documentation 

Within  a  given  program  or  project,  virtually  every  event  and  milestone  that  make  up  the  system’s 
life  span  can  benefit  from  visual  documentation.  Certain  media  are  more  suited  for  given  events  or 
milestones  in  the  RDT&E  process  depending  upon  the  requirement. 

The  following  outline  is  an  example  of  milestones  and  visual  media  products/documentation  that 
would  apply  at  those  designated  times. 


Milestone 

Purpose 

Products 

a. 

Identify/Define 

Requirement 

Briefing/Proposals 

Viewgraphs,  Slides 
Displays,  Illustrations 

b. 

Submit  Proposals/ 

Obtain  Funding 

Briefing/Presentations 

Viewgraphs,  Slides 

c. 

Research 

Document/Record  Data 

Use  in  Reports 

Still  Photography 
Viewgraphs,  Slides 
Illustrations 
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d.  Development 

e.  Testing/Evaluation 

f.  Modifications/ 
Follow-On  OT&E 

g.  Project  Completion 


Document/Record  Data 
Use  in  Reports 
Document/ Record  Data 
Use  in  Reports 
Document/Record  Data 
Use  in  Reports 
Final  Report/Historical 
Report 


Motion  Pictures,  Still 
Photos,  Videofilm 
Motion  Pictures,  Still 
Photos,  Videofilm 
Motion  Pictures,  Still 
Photos,  Videofilm 
Videofilm,  Viewgraphs 
Slides 


While  there  are  no  established  criteria  for  visual  documentation,  the  above  outline  cites  several  areas 
to  consider.  The  value  of  visual  records  to  RDT&E  programs  has  been  demonstrated  countless  times. 

10.3.3  Visual  Media  Products  and  Services 

The  following  visual  media  products  and  services  are  available  through  Code  962,  Technical 
Information  Division 

10.3.3.1  Graphics.  Illustrations,  architectural  renderings,  and  film  art  are  available  for  use  in  motion 
pictures  and  video,  brochures,  programs  and  flyers,  signs,  exhibits  and  displays,  computer  graphics, 
and  special  design  applications. 

10.3.3.2  Still  Photography.  This  covers  color  and  black  &  white  processes,  printing,  special 
large/oversize  black  &  white  technical  and  engineering  documentation,  underwater  applications,  and 
optical  instrumentation. 

10.3.3.3  Motion  Pictures  and  Television.  This  includes  color  and  black  &  white  motion  photography, 
motion  analyses,  television  technical,  engineering  and  documentation  underwater,  and  one-time  set¬ 
ups  for  experiments  and  tests.  Full  postproduction  services  are  offered  including  editing,  sound  recording, 
special  effects,  and  other  electronic  features.  Script  writing  from  concepts  to  final  narrative. 

10.3.3.4  Archives/Library.  Original  test  material  is  either  on  hand  or  can  be  secured  from  the  U.S. 
Navy  Photo  Center.  We  have  still  photo,  motion  picture,  and  video  records  of  previous  and  current 
programs  and  projects. 

10.3.3.5  Exhibits,  Displays  and  Special  Projects.  Any  special  display  can  be  created  from  sketches 
or  ideas.  These  can  be  two  or  three  dimensional. 

10.3.3.6  Presentations.  The  Visual  Media  Branch  can  create  and  assemble  a  media  package  for 
presentation  by  yourself  or  as  a  self-contained  program  for  distribution  to  other  activities. 

10.3.3.7  Audiovisual  Equipment  Loan  Pool.  The  service  is  available  to  all  Center  personnel.  The 
loan  pool  was  established  as  a  source  for  common  types  of  AV  equipment  and  accessories  that  are 
needed  for  short  term  loans.  Included  are  slide  and  motion  picture  projectors,  video  recorder  playback 
units,  VHS  self-contained  video  camcorders,  still  picture  cameras,  and  associated  accessories. 
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10.3.4  Guidelines  for  Viewgraphs 

Viewgraphs  are  the  principal  media  used  at  NOSC  for  briefings  and  presentations.  However,  35mm 
slides  can  be  produced  from  the  same  artwork,  if  required. 

The  Audiovisual  and  Publications  Branches  of  Technical  Information  working  in  concert  provide 
complete  services  from  layout  and  design  to  editorial  review.  All  viewgraphs  are  edited,  then  proofread 
after  completion.  TID’s  responsibility  is  to  ensure  all  viewgraphs  convey  the  information  while 
representing  NOSC  in  the  most  favorable  way. 

10.3.4.1  The  Guidelines.  The  following  viewgraph  preparation  guidelines  will  help  ensure  a  measure 
of  consistency  between  various  NOSC  presentations  and,  at  the  same  time,  give  your  presentations 
a  more  professional  look.  Figures  10.1  through  10.4  are  sample  viewgraph  formats. 

First,  always  remember  your  audience!  The  basic  requirement  for  a  viewgraph  is  that  it  be 
READABLE  by  everyone  in  the  audience.  To  meet  this  requirement,  the  message  on  the  screen  should 
be  conveyed  simply  and  quickly. 

Second,  work  toward  the  fewest  and  shortest  words  possible.  The  fewer  words  on  your  visual  the 
better  your  idea  will  be  understood.  Some  ways  to  get  concise,  clear  visuals  are  listed  here: 

a.  Cut  all  unnecessary  qualifiers  (words  or  phrases  that  modify,  limit,  or  qualify  other  words  or 
phrases).  Well-done  viewgraphs  have  NO  qualifiers— the  speaker  provides  them. 

b.  Cut  down  on  connectives  (and,  or,  for,  but,  yet  &  nor).  Instead,  commas  and  ampersands  (&) 
can  be  used  in  visuals  to  cut  down  on  words. 

c.  Limit  your  total  word  count  to  50  or  less.  This  doesn’t  include  the  title.  Remember,  however, 
to  keep  your  titles  short  and  meaningful.  Long,  rambling  titles  tend  to  introduce  long,  rambling 
messages. 

d.  Break  your  viewgraph  into  sections.  If  your  message  must  exceed  50  words,  put  each  section 
on  a  new  viewgraph.  Two  well-done  viewgraphs  are  always  more  understandable  than  one  that 
is  crowded  and  verbose. 

Following  the  above  guidelines  will  help  you  present  the  NOSC  story  in  the  most  favorable  light. 
In  addition  to  those  steps  mentioned  above,  the  graphics  section  uses  the  following  rules  when  preparing 
your  viewgraphs: 

a.  All  viewgraphs  have  a  red  title,  black  text,  and  a  clear  background. 

b.  Diagrams  and  graphs  use  blue  lines. 

c.  The  NOSC  logo  appears  in  the  upper  left  of  the  image  area. 

19.3.4.2  Scheduling.  To  ensure  a  quality  product,  the  graphics  section  needs  at  least  5  working  days 
from  date  received  in  graphics  to  date  delivered  back  to  you.  This  gives  us  enough  time  not  only  to 
edit  your  rough  copy,  but  also  to  proof  the  prepared  viewgraphs  and  fix  mistakes.  Rush  jobs  are 
sometimes  unavoidable,  but  editing  and  proofing  are  the  items  we  cut  when  you  need  a  job  quickly. 
Try  to  plan  your  presentations  with  enough  lead  time  to  let  us  give  you  the  best  product. 
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Figure  10.1.  Viewgraph  sample  1. 
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10.3.5  Planning  Considerations  for  Visual  Media 

Just  as  time  itself  is  transient  and  cannot  be  recaptured,  so  is  the  chance  for  visual  documentation. 
Time  passes  by  and  so  does  the  historical  record  unless  it  is  preserved  on  film  or  video.  The  visual 
records  of  World  War  II  and  other  modern  conflicts  have  proven  the  immeasurable  value  of  photographic 
documentation. 

The  Technical  Information  Division  (TID)  exists  to  provide  products  and  services  to  the  Center 
and  specifically  to  program  managers.  The  Visual  Media  Branch  of  TID  can  best  serve  programs  and 
projects  by  providing  documentation  of  key  events  and  milestones.  The  milestone  examples  listed 
previously  under  Visual  Documentation  (10.3.2)  should  all  be  considered  when  your  total  requirements 
for  the  project  are  defined. 

Bring  the  visual  media  staff  into  the  planning  process.  This  will  ensure  adequate  coverage  is  included 
formally  in  your  project  documentation. 

Contact  the  applicable  visual  media  staff  person  or  section  for  assistance  on  all  proposals. 

10.3.6  Contacts  for  Assistance 


Management 

Ken  Callahan 

x7817 

Visual  Info  Consultants 

A1  Woerner, 

Jim  Sundell 

x2577, 

x2041 

Graphics 

George  Galaich 

x6438, 

39 

Still  Photo 

Lutz  Winkler 

x6214 

Motion  Picture/ 

Television 

Stan  Follis 

x7741 

Photo  Officer  (Code  96) 

Roy  George 

x20 41, 
2577 

AV  Equipment 

Ernie  Santos 

x6885 

10.4  TECHNICAL  LIBRARY  SERVICES 
10.4.1  Library  Collections 

The  NOSC  Technical  Library  is  the  largest  technical  library  in  the  San  Diego  region,  excluding 
the  libraries  at  UCSD  and  San  Diego  State.  Materials  collected  are  in  those  disciplines  and  subject 
areas  pertinent  to  the  current  NOSC  mission.  Materials  are  located  in  the  bayside  and  topside  libraries 
according  to  the  number  of  scientists  and  engineers  working  in  particular  areas.  For  example,  underwater 
ordnance,  underwater  engineering,  and  marine  chemistry  materials  are  found  primarily  in  the  bayside 
library,  while  communications  and  artificial  intelligence  materials  are  in  the  topside  library. 

The  library  holds  all  major  reference  works,  indexes,  and  abstracts  in  the  physical  sciences, 
engineering,  and  life  sciences  pertinent  to  the  NOSC  mission.  Special  resources  include  an  extensive 
collection  of  maps  and  charts  located  in  the  topside  library  and  staffed  by  a  professional  map  librarian. 
The  bayside,  topside,  and  Hawaii  libraries  each  house  a  complete  and  cunent  collection  of  all  military 
and  federal  specifications,  standards,  handbooks,  and  drawings. 


10.4.2  Acquisitions  Policy 

New  titles  are  selected  for  purchase  by  NOSC  librarians  (frequently  in  advance  of  publication), 
and  also  by  library  users.  The  library’s  overhead  funds  are  used  for  most  book  purchases,  although 
funding  is  sometimes  requested  from  users  for  unusually  expensive  or  very  specialized  titles,  and  also 
desk  copies.  Codes  are  requested  to  fund  all  of  their  periodical  subscription  requirements,  although 
the  library  will  place  the  orders  and  provide  receipt  control. 

Program  managers  should  attempt  to  anticipate  their  need  for  both  specific  publications  and  subject 
coverage  by  consulting  with  NOSC  librarians.  Acquisition  of  publications  can  be  a  lengthy  process. 

The  library  is  the  internal  approval  control  point  for  all  Center  purchases  of  books,  publications, 
maps,  specification,  subscriptions,  etc.  The  library  will  determine  the  most  expeditious  means  of 
acquisition,  and  in  most  cases,  will  handle  purchase  of  publications  intended  for  exclusive  use  by  a 
project  office.  Such  publications  will  be  cataloged  in  the  library’s  system,  but  will  also  be  identified 
as  in  use  for  an  indefinite  period  by  the  requesting  project  office. 

Publications  that  are  not  available  for  purchase  may  usually  be  borrowed  from  other  libraries 
or  photocopied.  The  NOSC  library  borrows  from  not  only  local  libraries,  but  also  from  Department 
of  Defense,  corporate,  public,  and  university  libraries  throughout  the  United  States  and  Great  Britain. 


10.4.3  Library  Access 

The  library  is  accessible  by  all  NOSC  employees  and  also  NOSC  contractors.  Materials  may  be 
borrowed  only  by  NOSC  employees.  NOSC  contractors  may  use  materials  on  hand;  materials  in  use 
will  not  be  recalled. 

Program  managers  should  discuss  the  technical  information  requirements  of  their  contractors  with 
the  NOSC  librarian  for  the  purpose  of  planning  appropriate  library  access  and  services.  Contractor 
access  to  the  library’s  technical  report  collection  must  be  on  a  need-to-know  basis.  Procedures  are  detailed 
in  NOSCINST  5070. IB,  paragraph  5. 


10.4.4  Reference/Literature  Search  Services 

The  library  offers  reference  service  by  professional  librarians  ranging  from  locating  specific 
handbook  type  information  to  compiling  comprehensive  subject  bibliographies.  Online  access  is 
maintained  to  all  available  commercial  and  government  databases  pertinent  to  NOSC’s  areas  of  interest. 
Librarians  search  these  databases  at  no  charge  to  Library  users. 

The  library  maintains  secure  online  links’  to  the  databases  of  the  Defense  Technical  Information 
Center  (DTIC).  NOSC  scientists  and  engineers  are  required  to  request  searches  of  the  DTIC  databases 
when  beginning  new  projects  in  accordance  with  DoD,  Navy,  and  NOSC  instructions  for  the  purpose 
of  verifying  that  their  projects  are  not  duplicating  other  efforts. 

10.4.5  Library  Publications 

The  library  disseminates  a  list  of  its  periodical  holdings,  a  monthly  list  of  new  publications  acquired, 
and  various  current  awareness  listings.  Customized  current  awareness  listings  may  be  requested  to  cover 
specific  project  interests  at  no  cost  to  the  requester. 


10-18 


10.4.6  Special  Projects  and  Services 


Installation  of  the  Library's  new  automated  system  which  will  provide  online  access  to  the  library’s 
catalogs  is  well  underway.  Included  is  a  NOSC  Database  of  Databases  which  identifies  libraries  and 
databases  at  NOSC  that  are  outside  the  NOSC  Technical  Library.  Program  managers  are  requested 
to  register  brief  descriptive  information  about  their  project  databases  with  the  technical  library.  In 
addition,  program  managers  are  requested  to  consult  with  the  NOSC  librarian  before  establishing  new 
databases  and  libraries. 

Program  managers  may  request  library  support  for  contractor  review  of  documentation. 

The  library  offers  a  presentation  on  library  services,  and  also  one  specifically  on  its  literature  search 
services.  These  presentations  are  geared  toward  the  specific  subject  interests  of  each  group. 

Finally,  Tables  10.1  through  10.4  present  further  specific  information  on  library  contacts,  VSMF 
holdings,  accessible  retrieval  systems,  and  SDI  services.  The  NOSC  library  request  for  literature  search 
form  is  shown  in  Figure  10.5. 


10.5  EFFECTIVE  PRESENTATIONS 

Presentations  come  with  the  territory;  presentation  demands  are  made  on  project  managers  at 
the  Naval  Ocean  Systems  Center.  Technical  presentations  or  briefings  are  often  vital  to  obtaining  support, 
selling  your  ideas,  or  defining  the  scope  and  requirements  of  your  project*.  The  professional  image 
you  portray,  how  you  organize  and  present  material,  the  visual  aids  you  choose  to  help  you,  how  you 
deliver  the  message,  can  all  mean  success  or  failure  in  your  presentations. 

Your  oral  presentation  may  be  given  in-house  to  your  peers  or  management.  When  you  gain  a 
reputation  for  giving  effective  presentations,  you’ll  benefit  from  a  “halo”  effect  that  carries  over  to 
other  job  activities.  If  your  presentation  is  given  to  others,  the  impression  you  make  on  your  audience 
will  also  reflect  the  image  of  the  Naval  Ocean  Systems  Center.  If  you’re  well-organized,  sharp,  concise, 
and  self-confident,  NOSC  will  be  regarded  as  an  efficient  and  professional  organization. 

The  few  minutes  of  your  presentation  may  reflect  years  of  background,  months  of  study  and  data 
accumulation,  and  days  or  maybe  even  weeks  of  preparation.  In  many  instances,  those  few  minutes 
can  affect  management  decisions  and  the  expenditure  of  large  sums  of  money. 

As  a  result,  presentation  skills  are  important  to  you  as  a  project  manager  and  each  briefing  requires 
careful  and  intelligent  attention. 

The  NOSC  Presentation  Workshop  is  given  periodically  for  NOSC  personnel.  It  has  strong  support 
from  upper  management.  It  was  designed  to  not  only  give  you  individualized  instruction  in  speaking 
and  presentation  techniques,  but  to  familiarize  you  thoroughly  with  the  audiovisual  and  graphics  support 
you  have  available  here  at  the  Center. 

The  following  publications  are  recommended  for  those  preparing  presentations: 

Effective  Business  and  Technical  Presentations,  by  George  L.  Morrisey 

How  to  Prepare,  Stage,  and  Deliver  Winning  Presentations,  by  Thomas  Leech. 


10-19 


Table  10.1.  Whom  to  contact  at  the  NOSC  libraries 


TOPSIDE  LIBRARY 


Reference/Literature  Searches/Ordering 

Jo  Walsh 

x6623 

Marcia  Whipple 

x6623 

Circulation 

Linda  Kerr 

x6621 

Bernice  Kuntz 

x6622 

Maps,  Charts,  Specs,  and  Standards 

Val  Danes h 

x6623 

Periodicals 

Jeanie  Casto 

x6625 

BAYSIDE  LIBRARY 

Reference/ Literature  Searches/T ranslations/Ordering/ 

Kathy  Wright 

x6171 

Current  Awareness 

Diane  Soblick 

x6171 

Helen  Cook 

x6171 

Circulation 

Jan  Rutledge 

x2357 

Interlibrary  Loan 

Yoli  Kerr 

x6155 

NOSC  DATABASE  OF  DATABASES 

Kathy  Wright 

X6171 

LIBRARY  CONSULTING  SERVICE  FOR 

Kathy  Wright 

X6171 

DATABASES 

VISITING  LIBRARY  SERVICE 

Kathy  Wright 

X6171 

ELECTRONIC  MAIL 

Bayside  Library 

BAYLIB 

Topside  Library 

TOPLIB 
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Table  10.2.  NOSC  Technical  Library 
VSMF  holdings 


TOPSIDE  LIBRARY 

Military  &  Federal  Specifications  &  Standards 

Military  Standardization  Package 

Navy  International  Standardization  Documents 

Industry  Standards 
ANSI 

Electrical  &  Electronic 
Information  Systems 
Mechanical 
ASME  Standards 
EIA  Standards 

Vendor  Catalog  Data 

Design  Engineering  Service 
Documentation  Service 
Integrated  Circuit  Parameter  Retrieval 
Semiconductor  Parameter  Retrieval 


Procurement 

Defense/Federal  Acquisition  Regulations 
Telecommunications 

CCITT.  International  Telegraph  &  Telephone 
Consultative  Committee  Recommendations 


BAYSIDE  LIBRARY 

Military  &  Federal  Specifications  &  Standards 
Military  Standardization  Package 

Industry  Standards 
ANSI 

Electrical  &  Electronic 
ASTM  Standards 
AWS  Standards 
EIA  Standards 
NAS  (AIA) 

SAE — Aerospace  Material  Specifications 

Vendor  Catalog  Data 

Design  Engineering  Service 
Documentation  Service/High  Tech 
Integrated  Circuit  Parameter  Retrieval 
Marine  Engineering  Service 
Metric  Design  Service 
Semiconductor  Parameter  Retrieval 


May  1985 
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Table  10.3.  Online  information  retrieval  systems 
accessible  by  the  NOSC  Technical  Library 

DTIC DROLS.  The  Defense  Research  On-Line  System  (DROLS),  maintained  by  the  Defense  Technical 
Information  Center,  provides  classified  (through  secret)  access  to  past,  current,  and  planned  work 
sponsored  by  the  Department  of  Defense.  DROLS  consists  of  three  databases:  Technical  Reports,  Work 
Units  (1498s),  and  industrial  Independent  Research  and  Development  Summaries. 

LMARS.  The  NOSC  Library  Management  and  Retrieval  System  (LMARS)  is  an  automated  retrieval 
system  developed  inhouse  to  provide  bibliographic  control  of  the  library’s  technical  reports  collection. 
LMARS  generates  a  microfiche  catalog  of  the  library  s  holdings.  The  catalog  consists  of  seven  indexes: 
corporate  author,  personal  author,  title,  report  number,  contract  number,  accession  number,  and  subject. 
The  LMARS  also  generates  the  library’s  list  of  new  publications,  recall  notices,  custom  bibliographies, 
and  Current  Alert  notices  of  new  acquisitions  in  specific  subject  areas. 

NASA  RECON.  Provides  access  to  over  2  million  technical  reports,  journal  articles,  books,  conference 
proceedings,  and  other  publications  in  the  areas  of  aerospace  and  related  technologies. 

DIALOG.  Provides  access  to  over  250  databases  in  all  fields,  many  of  which  correspond  to  printed 
equivalents.  Some  of  those  most  frequently  searched  are  COMPENDEX  (Engineering Index),  OCEANIC 
ABSTRACTS,  CHEMICAL  ABSTRACTS,  SCISEARCH  (Science  Citation  Index),  DEPARTMENT 
OF  ENERGY  DATABASE,  DISSERTATION  ABSTRACTS,  and  INSPEC  (Physics  Abstracts,  Electrical 
and  Electronics  Abstracts,  Computer  and  Control  Abstracts,  and  IT  Focus). 

ORBIT.  Provides  access  to  over  70  databases,  including  COLD,  a  database  of  technical  reports,  papers, 
articles  and  books  compiled  and  maintained  by  the  Army  Cold  Regions  Research  and  Engineering 
Laboratory  and  the  Library  of  Congress. 

BRS.  Provides  access  to  more  than  70  databases,  including  RBOT,  a  database  for  bibliographic  citations 
on  robotics;  and  TECHDATA,  an  online  index  to  specifications,  standards,  and  manufacturing  catalog 
information. 

WILSONLINE.  Provides  access  to  indexes  such  as  Applied  Science  &  Technology  Index ,  Business 
Periodicals  Index,  and  Readers’  Guide  to  Periodical  Literature. 

VU/TEXT.  Provides  access  to  information  in  newspapers  such  as  The  Washington  Post,  Philadelphia 
Inquirer,  and  Chicago  Tribune. 

QL  SEARCH.  Provides  access  to  the  Arctic  Science  and  Technology  Information  System  produced 
by  the  Arctic  Institute  of  North  America  at  the  University  of  Calgary,  Alberta. 

OCLC.  An  online  union  catalog  of  approximately  12  million  records  for  books,  serials,  and  maps, 
owned  by  3,600  member  libraries. 


Table  10.4.  SDI  services  offered  by  the  NOSC  Technical  Library 

SDI:  Selective  dissemination  of  information:  periodic  announcement  of  new  publications  in  specific 
subject  areas 


INHOUSE 


-CURRENT  ALERT 
-CURRENT  AWARENESS 


EXTERNAL 

-DTIC 

-DIALOG 

-BRS 

-ORBIT 

-NASA 


new  technical  reports  received  in  the  libraries 
bibliographies  of  new  publications  in: 

applied  mathematics 
the  arctic 

artificial  intelligence 

command  and  control 

communication  satellites 

communication  systems  and  equipment 

communication  theory 

data  communications 

display  systems 

feedback  and  control  theory 

fiber  optics 

image  processing 

matrix  materials 

microelectronics 

microprocessors 

numerical  analysis 

oceanography 

optical  communications 

optics 

radar 

radio  noise 

reliability 

robotics 

semiconductors  and  transistors 
software  engineering 

software  quality,  reliability  and  documentation 
VHSIC 


Individually  tailored  SDI  searches  may  be 
established  with  any  of  these  external  services. 


October  1985 
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REQUEST  FOR  LITERATURE  SEARCH  Code _ 

NOSC  TECHNICAL  LIBRARY  Phone _ (  }  CaJ,  me  for  pickup 

(  )  Mail  search  to  me 

Date _  Date  Required _ 

SEARCH  TOPIC 

Please  provide  a  narrative  statement  of  your  topic.  Define  any  terms  that  may  have  special  meaning 
in  your  request.  Indicate  areas  to  be  excluded.  Include  significant  phrases,  synonymous  terms,  relevant 
authors,  companies,  agencies,  etc.,  or  pertinent  citations.  BE  AS  SPECIFIC  AS  POSSIBLE. 


SEARCH  SPECIFICATIONS 

Specificity:  Exhaustive _  Specific _  Few  key  articles _ 

Time  coverage: _ (no.  of  years) 

Language:  Ail _  Eng.  only _ 

Highest  classification: _ 


DATA  BASES  TO  BE  SEARCHED 

_ NOSC  Library  Holdings 

DTIC: 

_ Technical  Reports 

_ Work  Units  (1498s) 

_ Industrial  IR&D 

_ Program  Planning 


Periodical/open  literature: 

_ DIALOG 

_ BRS 

_ ORBIT 

_ NASA 

_ Other  (please  specify) 


Figure  10.5.  Request  for  NOSC  library  literature  search  form. 


The  tables  that  follow  present  a  series  of  ready  reference  materials  that  are  concise  but  thorough 
enough  to  be  of  significant  help  in  preparing  effective  presentations.  Use  them. 

Table  10.5.  Criteria  for  selection  of  resource  material 
Table  10.6.  How  to  plan  and  organize  your  presentation 
Table  10.7.  Audience  analysis 
Table  10.8.  The  presentation  outline 
Table  10.9.  How  to  give  your  presentation 
Table  10.10.  Preliminary  arrangements  checklist 
Table  10.11.  Visual  aids 
Table  10.12.  Odds  and  ends 


Table  10.5.  Criteria  for  selection  of  resource  material 

What  is  the  OBJECT  or  PURPOSE  of  the  presentation? 

What  kind  of  AUDIENCE  ACTION  or  RESPONSE  is  required? 

What  MUST  be  said  to  reach  objectives? 

What  is  the  BEST  way  to  say  it?  What  method  to  use? 

What  amount  of  DETAIL  is  necessary? 

What  material  should  be  WITHHELD,  but  available  for  question/answer  period? 
Submit  all  resource  materials  to  the  “WHY”  test. 


Table  10.6.  How  to  plan  and  organize  your  presentation 

DETERMINE  PURPOSE 

State  in  one  concise  sentence 

ANALYZE  YOUR  AUDIENCE 

Size,  attitude,  background,  relationship 

PREPARE  PLAN 

Main  ideas  or  concepts,  supporting  material 

SELECT  RESOURCE  MATERIALS 
Submit  all  to  the  “WHY”  test 

ORGANIZE  WELL 

Opening,  body,  close 

PRACTICE 


Rehearse,  rehearse,  rehearse 


Table  10.7.  Audience  analysis 


How  much  do  they  know  about  the  subject? 

Are  they  at  the  decisionmaking  level? 

What  language  will  they  best  understand: 

Technical,  business,  financial,  everyday  English,  or  what? 

Are  there  leaders  in  the  group  who  could  sway  the  rest? 

Should  I  address  myself  to  the  whole  group,  or  only  certain  ones? 

What  are  their  reasons  for  attending  my  presentation? 

What  information  or  technique  is  likely  to  gain  their  attention? 

What  information  or  technique  is  likely  to  get  negative  reactions? 

Audience  attitude?  Friendly,  unfriendly,  etc. 

Will  they  be  in  a  hurry  to  conclude? 

Is  there  likely  to  be  opposition,  or  even  debate? 

Does  anyone’s  face  need  to  be  saved? 

Is  there  likely  to  be  a  bias,  either  pro  or  con? 


Table  10.8.  The  presentation  outline  (1  of  2) 


Title  or  Subject: _ 

Purpose  (state  in  one  clear,  concise  sentence) 


Opening: 


Information  necessary  to  support  the  main  idea: 


Table  10.9.  How  to  give  your  presentation 


THE  TOTAL  IMPRESSION 

Professional  —  Confidence,  Alertness,  Poise 

THE  LOOK 

Appearance  —  Affects  attitudes,  answers  questions 
Affects  opinion,  reinforces  feelings 
Dress  like  occasion  demands 
Be  comfortable 
Nothing  distracting 

LECTERN  PRESENCE 

Bearing  —  Posture,  movements 

Hand  gestures  —  Relax,  never  make  gestures  consciously,  key  is  natural, 
don’t  play  with  items  you  hold 
Body  gestures  —  Move  with  purpose,  avoid  pacing 
Eye  contact  —  Use  it 
Lectern  —  Establishes  formal  relationship 

VOICE 

Volume,  rate,  modulation 

Pause  —  Effective  means  of  emphasis 

Watch  filler  words/pet  phrases 

Be  aware  —  Enunciation/voice  drop 

DEVELOP  STYLE 

Personality  —  warmth/sincerity/enthusiasm 

Never  imitate 

Discover  personal  strength 

MEMORY  AIDS 

Avoid  hazards  of  memorizing 
Visuals  keep  “on  track” 

Use  notes  if  more  comfortable 


Table  10.10.  Preliminary  arrangements  checklist 


GENERAL  INFORMATION 

Presenter  _ 

Subject  _ 

Audience _ 

Date _ 

Security  Classification  _ 

ROOM 

□  Room  Reserved 

□  Arrangement 

□  Chairs 

□  Tables 

□  Lighting 

G  Ventilation 
G  Distractions 
G  Other  _ 

PRESENTATION  MATERIAL 

□  Visual  Aids 
G  Film/Tapes 
G  Hardware/Models/Demonstrations 

□  Handouts  (quantity) 

□  Other  _ 

EQUIPMENT  (Test  Everything) 

□  Placement 

□  Accessories  (bulb,  extension  cord,  etc.) 

□  Sound  System 
G  Lectern 

G  Pointer 

□  Marker/Chalk 

□  Other  _ 

SUPPLIES 

□  Tablets 
G  Pencils 

G  Name  Cards 

G  Other  _ 

PROVISIONS 

□  Refreshments 
G  Breaks 

G  Other  _ 


Number 
Time _ 
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Table  10.11.  Visual  aids 


WHY  VISUALS? 

People  are  visual-minded 
Retention  is  increased 
Visualization  encourages  organization 
Puts  you  in  action 

You  and  the  Audience  are  side  by  side 
Misunderstandings  are  less  likely  to  occur 

VISUALS  SHOULD 

Illustrate 
Focus  attention 
Clarify 

GENERAL  RULES 

Keep  them  simple 
Make  them  readable 
Use  key  words 
Use  consistent  style 
No  more  than  7  items 

TYPES  OF  VtSUALS 

Boards 

Pictorials 

Charts 

Objects/Models 
Projections 
Handout  Materials 
Auditory  Aids 


Table  10.12.  Odds  and  ends 


SITUATION 

Where  given  —  never  ignore  details,  arrive  early,  check  facilities,  check  equipment  for  visuals 

Time  limit  —  no  excuse  for  running  over 

Place  on  Program  —  make  adjustments  accordingly 

Handout  Material  —  pass  out  after  talk 

ANALYZING  AUDIENCE  FEEDBACK 
Remain  objective 

Don’t  continue  without  audience  contact 

AUDIENCE  RETENTION 

Strong  introduction  and  conclusion 
Need  for  closing  phrases 

QUESTION  AND  ANSWER  PERIOD 

Advantages  —  message  reinforced,  valuable  feedback 
Planning  —  allow  time,  prepare  audience,  prepare  yourself 
Handling  —  getting  it  started,  heard  by  all,  give  everyone  a  chance 

Analyzing  —  spot  loaded  questions,  divide  complex  questions,  accept  nonquestions,  dismiss 
irrelevant  questions,  handling  long-winded  questions 
Answer  directly 
Stick  to  your  specialty 
Keep  it  moving 

Concluding  —  stop  while  you’re  ahead,  anticipate  the  end,  close  with  summary  statement 


HARDWARE 

PRODUCT 

ASSURANCE 
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SECTION  11 

HARDWARE  PRODUCT  ASSURANCE 
G.  Thirkill,  Code  9202 


11.1  1NTRODUCT  ON 


11.1.1  References 

See  subsection  11.10. 

11.1.2  Outline 

Introduction 

References 

Outline 

Summary 

Hardware  Product  Assurance  Overview 
Reliability/Maintainability  Assurance 
Definitions 

Reliability  Program  Policy 
Maintainability  Program  Policy 
Reliability  Assurance  Program  Elements 
Maintainability  Assurance  Program  Elements 
System  Safety  Assurance 
Definitions 

System  Safety  Program  Policy 
System  Safety  Assurance  Program  Elements 
Human  Factors 
Definitions 

Human  Factors  Policy 
Human  Engineering  Program  Elements 
Quality  Assurance 
Definitions 

Quality  Assurance  Program  Policy 
Quality  Assurance  Program  Elements 
Why  Inspect  at  the  Piece  Part  Level? 
Configuration  Management 
Definitions 

Configuration  Management  Policy 
Configuration  Management  Program  Elements 
Design  Assurance 
Definition 

Design  Assurance  Policy 
Design  Assurance  Program  Elements 
Integrated  Logistic  Support 
Definition 


Integrated  Logistic  Support  Program  Policy 
Integrated  Logistic  Support  Program  Elements 
DoD/Navy/NOSC  Product  Assurance  Policy  Overview 
Reliability  and  Maintainability  Policy  Directives 
System  Safety  Policy  Directives 
Human  Factors  Policy  Directives 
Quality  Assurance  Policy  Directives 
Configuration  Management  Policy  Directives 
Design  Review  Policy  Directives 
Design  Documentation  Policy  Directives 
Integrated  Logistic  Support  Policy  Directives 


11.1.3  Summary 

See  below. 


11.2  HARDWARE  PRODUCT  ASSURANCE  OVERVIEW 

Product  assurance  includes  those  various  engineering  and  technical  management  disciplines  that, 
when  coordinated  and  integrated  with  the  design  effort,  enhance  the  suitability  of  an  item  of  equipment 
for  production  and  Fleet  use.  Figure  11.1  illustrates  what  product  assurance  activities  take  place  during 
the  various  life-cycle  phases.  The  primary  objective  of  a  product  assurance  program  is  to  ensure  that 
products  supplied  to  the  Fleet  will  achieve  a  level  of  overall  quality  consistent  with  the  operational 
requirements.  In  order  to  meet  this  primary  objective,  the  hardware  product  assurance  program  is  planned 
and  structured  to  provide  the  following: 

Participation  with  the  designer  in  developing  reliable,  maintainable,  and  safe  systems/equipment 

Assurance  that  the  systems/equipment  Fleet  logistic  support  requirements  have  been  fully  identified 
and  integrated  and  that  those  requirements  will  be  satisfied 

Assurance  that  systems/equipment  designs  are  producible  and  are  fully  disclosed  and  documented 
for  production 

Assurance  that  those  systems/equipment  items  which  are  fabricated  in  production  conform  to  the 
engineering  drawings  and  specifications  and  are  of  high  overall  quality 

Assurance  that  those  systems/equipment  items  which  are  introduced  into  the  Fleet  are  fully  supported 
throughout  their  life  cycle. 

The  specific  concerns  of  hardware  product  assurance,  therefore,  are  the  following: 

Reliability 
Maintainability 
Availability 
System  safety 
Human  compatibility 
Quality 

of  design,  design  documentation,  and  produced  products 
Configuration  integrity 
Producibility 
Spares  reprocurability 
Logistic  supportability. 


•w 
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Figure  11.1.  Product  assurance  activity  during  various  life-cycle  phases. 


These  concerns  have  been  ordered  in  a  set  of  engineering  and  technical  management  disciplines 
which  make  up  the  typical  hardware  product  assurance  program.  These  disciplines  are: 

Reliability/  maintainability  assurance 
System  safety  assurance 
Human  factors 
Quality  assurance 
Configuration  management 
Design  assurance 
Integrated  logistic  support. 

The  above  disciplines  will  be  outlined  in  detail  in  subsections  1 1 .3  through  1 1 .9.  Each  of  these 
subsections  will  begin  with  a  definition  of  the  terms  associated  with  the  subject  and  will  include  a  review 
of  the  most  significant  policy  directive(s)  that  guides  the  discipline  activity.  The  major  task  elements 
which  typically  are  considered  when  planning  the  requirements  for  the  program  (e.g.,  the  quality  assurance 
program)  are  identified  and  described;  an  expanded  view  of  certain  program  elements  (e.g.,  environmental 
stress  screening)  is  provided.  Technical  Document  432,  Product  Assurance  Requirements  Guide  for 
Naval  Ocean  Systems  Center  Projects,  provides  additional  background  regarding  these  elements  and 
provides  recommended  contractual  requirements  statements  which  may  be  used  in  contract  statements- 
of-work. 

Subsection  11.10  provides  an  overview  and  brief  description  of  the  major  policy  directives  which 
influence  product  assurance.  TD  432  provides  a  more  complete  list  of  product  assurance  directives. 


11J  RELIABILITY/MAINTAINABILITY  ASSURANCE 

m 

11.3.1  Definitions 


11.3.1.1  Reiiabiiity/Maintainability  Assurance.  This  is  the  continuing  analysis  and  monitoring  of 
system/equipment  design,  operation,  and  maintenance,  throughout  its  life-cycle,  to  assure  that  it  performs 
satisfactorily  under  the  required  conditions  and  for  the  required  period  of  time. 

11.3.1.2  Reliability.  Reliability  is  the  extent  to  which  a  system/equipment  is  capable  of  performing 
its  intended  function  under  stated  conditions  without  failure.  The  measurement  of  reliability  is  expressed 
two  ways:  mean-time-between-failures  (MTBF)  and  probability  of  success  (R). 

a.  Mean-Time-Between-Failures.  MTBF  is  the  total  functioning  life  of  a  population  of  a 
system/equipment  divided  by  the  total  number  of  failures  within  the  population  during  a 
particular  measurement  interval.  Can  be  expressed  in  time  (hours),  cycles,  miles,  events,  or 
other  measure  of  life  applicable  to  continuous  or  intermittently  operated  systems/equipment. 

b.  Probability  of  Success.  R  is  the  probability  that  an  item  will  perform  its  intended  function 
for  a  specified  time  interval  or  mission.  It  is  expressed  as  a  decimal  value  and  is  always  with 
an  associated  confidence  level,  and  it  is  appropriate  for  use  in  connection  with  item’s  for  which 
operating  time  is  undefinable  (e.g.,  explosive  devices,  rockets,  and  torpedoes). 

11.3.1.3  Maintainability.  Maintainability  is  the  ability  of  a  failed  system/equipment,  based  on  its 
design  characteristics,  to  be  restored  to  operation.  This  is  expressed  by  mean-time-to-repair  (MITR),  which 
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is  the  total  corrective  maintenance  time  performed  on  a  population  of  a  system/equipment  divided 
by  the  total  number  of  corrective  maintenance  actions  performed.  It  is  usually  expressed  in  hours. 

11.3.1.4  Supportability.  This  is  the  ability  to  satisfy  material  and  administrative  requirements  associated 
with  restoring  the  operation  of  a  failed  system/equipment. 

11.3.1.5  Availability.  Availability  is  the  measure  of  the  degree  to  which  a  system/equipment  is  in 
an  operable  and  committable  state  at  the  start  of  a  mission  which  is  called  for  at  an  unknown  time. 

a.  Inherent  availability  (Ai)  is  the  measure  of  a  system’s/equipment’s  performance  predicated 
on  the  inherent  design  factors  of  reliability  and  maintainability. 

..  MTBF 

Ai  =  _  . 

MTBF  +  MITR 

b.  Operational  availability  (Ao)  represents  the  expected  percentage  of  time  that  a  system/equipment 
will  be  ready  to  perform  satisfactorily  in  an  operating  environment. 


Ao  = 


Ao  = 


UPTIME 

UPTIME  +  DOWNTIME 
MTBF 

MTBF  +  MDT 


(basic  expression); 


MDT  =  mean  downtime  (includes  both  maintainability  and  supportability  factors). 

NAVMATINST  3000.2  establishes  Ao  as  the  primary  measure  of  Navy  system/equipment 
readiness.  NAVSEA  requires  that  Ao  be  reported  relating  to  the  mission  profile. 

11.3.2  Reliability  Program  Policy  (NAVMATINST  3000.1A) 

11.3.2.1  Objectives  An  objective  accomplishes  the  following: 

a.  Reaffirms  the  close  relationship  between  good,  conservative  design  and  product  reliability 

b.  Redirects  reliability  program  emphasis  towards  the  engineering  and  manufacturing  specifications, 
disciplines,  and  controls  by  which  reliable  systems/equipment  are  designed  and  produced. 

11.3.2.2  Policy  The  following  are  policy  considerations. 

a.  Reliability  is  as  important  as  functional  performance. 

b.  DSARC  1  (release  to  validation  phase  development):  the  DCP  to  address  reliability  requirements 

c.  DSARC  II  (release  to  full-scale  development):  the  DCP  to  state  that  reliability  will  be  by  design, 
not  just  left  to  chance:  the  mission  profile  will  be  defined  and  quantitative  reliability  requirement 
will  be  established. 

d.  DSARC  III  (release  to  production):  the  DCP  is  to  include  a  statement  of  reliability  achievement 
with  explanation  of  any  shortfalls  and  planned  corrective  actions. 

e.  Reliability  requirements  are  to  be  included  in  all  planning  and  procurement  documents  and 
are  to  be  a  major  factor  in  the  source  selection  and  contracting  process.  Where  appropriate, 
reliability  and  quality  incentives  should  be  included  in  contracts. 


11-7 


11.3.2.3  Technical  Requirements  The  technical  requirements  are  noted  below. 

a.  Specific  engineering  (i.e.  design)  and  manufacturing  disciplines  to  be  invoked.  Examples  would 
include: 

MIL-STD-454,  Standard  General  Requirements  For  Electronic  Equipment 
MIL-STD-188,  Military  Communication  System  Technical  Standard 
MIL-MDBK-5C,  Metallic  Materials  and  Elements  for  Aerospace  Vehicle  Structures 
MIL-MDBK-251,  Reliability/Design  Thermal  Applications 

MIL-E- 16400,  Electronic  Interior  Communication  and  Navigation  Equipment,  Naval  Ship 
and  Shore,  General  Specification  For 

MIL-STD-275,  Printed  Wiring  for  Electronic  Equipment 

MIL-P-55110,  Printed-Wiring  Boards 

NAVMAT  P4855-1,  Navy  Power  Supply  Reliability 

NAVMAT  P4855-2,  Design  Guidelines  for  Prevention  and  Control  of  Avionic  Corrosion 

WS-6536,  Procedures  and  Requirements  for  Preparation  and  Soldering  of  Electrical 
Connections 

b.  Explicit  reliability  program  requirements  to  include: 

Mission  profile  definition 

Allocation  of  numerical  reliability  requirements 

Parts  and  materials  program 

Conservative  parts  and  materials  derating  criteria 

Electrical,  mechanical,  and  thermal  stress  analyses 

Failure  modes,  effects,  and  criticality  analysis 

Sneak  circuit  analysis 

Worst  case  analysis  of  tolerance  buildup 

c.  Integrated  test  program  to  assess  reliability  growth  prior  to  reliability  demonstration 

Qualification  testing  to  environmental  extremes 

Acceptance  testing  to  mission  profile  environmental  conditions,  following  burn-in.  Use 
of  environmental  stress  screening  (e.g.  NAVMAT  P9492,  Navy  Manufacturing  Screening 
Program;  NAVSEANOTE  3900) 

d.  Reliability  tests,  where  warranted: 

Reliability  development/ growth  tests  where  uncertainty  exists  (e.g.,  new  technology  devices, 
highly  complex  equipment  items) 

Reliability  demonstration  of  production  prototype  units 

e.  Continuous  assessment  of  reliability  in  design  and  testing 

f.  Failure  reporting,  analysis,  and  corrective  action 

Failures  recorded  in  formal  reporting  and  tracking  system 


Failures  analyzed  to  identify  cause 
Failures  corrected  to  prevent  recurrence 
g.  Control  over  contractor  programs  and  progress,  through: 

Design  reviews  (PDR,  CDR,  DCR) 

Certification  of  test  results 
Submittal  of  reports 
Technical  audits  (e.g.  POS). 

11.3.3  Maintainability  Program  Policy 

11.3.3.1  NAVSEA1NST  3900.2  This  instruction  sets  out  the  following  relationships  and  requirements. 

a.  Reaffirms  the  inseparable  relationship  between  material  design  and  reliability  and  maintainability 
(R&M) 

b.  Directs  the  program  emphasis  to  the  engineering  and  manufacturing  specifications,  disciplines, 
and  controls 

c.  R&M  requirements  shall  be  considered  equal  to  functional  performance. 

d.  R&M  requirements  shall  be  addressed  as  a  major  issue  at  the  DSARC  I,  II,  and  III  reviews. 

e.  Qualitative  maintainability  requirements  (MITR)  are  to  be  established  for  systems/equipment. 

f.  R&M  requirements  are  to  be  included  in  all  procurement  documents  and  contracts  for  new 
equipment  development.  Where  appropriate,  R&M  incentive  clauses  are  to  be  used. 

g.  R&M  programs  are  to  be  established  in  connection  with  acquisitions  and  will  include 

Maintainability  test  program 

Problems  to  be  recorded  in  a  formal  reporting  and  tracking  system 

Problems  to  be  analyzed  to  determine  necessary  corrective  action 

Corrective  action  to  be  taken  until  MITR  requirement  satisfied. 

Control  over  contractor  programs  through  design  reviews,  certification  of  test  results, 
submittal  of  reports,  and  technical  audits 

R&M  allocations  and  predictions 

Logistics  support  analysis  to  be  conducted. 

h.  QAP  200  (superseded  by  OD  46574B)  to  be  invoked  in  engineering  agent  tasking  documents. 

11.3.3.2  NAVELEXINST  4858.3  This  instruction  sets  out  the  following  requirements. 

a.  Realistic  qualitative  and  quantitative  (MITR)  maintainability  design  requirements  are  to  be 
specified  which  strike  an  optimum  balance  between  logistic  support  capability,  potential  life¬ 
cycle  costs,  and  a  fully  maintainable  system. 


b.  Maintainability  requirements  are  to  be  included  in  all  planning  and  procurement  documents. 

c.  Maintainability  program,  citing  appropriate  elements  of  M1L-STD-470,  is  to  be  established 
including: 

Program  plan 
Maintainability  predictions 
Program  and  design  reviews 
Maintainability  testing 
Appropriate  data  items. 

11.3.3.3  NAVAR1NST  13070.2B  This  instruction  sets  out  the  following  objectives  and  requirements. 

a.  Define  mission  and  environmental  profiles. 

b.  Establish  quantitative  maintainability  performance  objectives. 

c.  Establish  maintainability  program  incorporating  requirements  of  MIL-STD-470  as  applicable 
to  the  project  phase.  Full-scale  development  phase  would  include: 

Program  plan 
Design  criteria 
Subcontractor  control 
Program  reviews 

Data  collection,  analysis,  and  corrective  action 
Modeling/  Allocations/Predictions 
FMEA  (maintainability) 

Inputs  to  LSA 
Demonstration. 

d.  Ensure  maintainability  requirements  are  met. 


11.3.4  Reliability  Assurance  Program  Elements 


11.3.4.1  Program  Planning,  Monitoring,  and  Control.  These  elements  are  discussed  below: 

a.  Program  planning  —  Definition  of  the  reliability  program  (MIL-STD-785)  in  a  plan  which 
identifies  and  describes  all  program  monitoring,  control,  design,  analysis,  evaluation,  test, 
demonstration,  and  documentation  elements 

b.  Subcontractor /Supplier  monitoring  and  control  —  Monitoring  by  the  prime  contractor  of  all 
subcontractor/supplier  reliability  program  efforts  and  ensuring  compliance  with  the  overall 
program  requirements 

c.  Program/Design  reviews  —  Review  of  the  reliability  program  progress  at  specified  points  in 
time  and  conduct  of,  as  minimum,  preliminary  and  critical  design  reviews  per  MIL-STD-1521 

d.  Failure  reporting  —  Establishment  of  a  closed-loop  failure  reporting  system  providing  for  the 
analysis  of  and  correction  of  failures 

e.  Failure  review  board  —  Review  of  significant  system/equipment  failures,  failure  trends,  and 
the  status  of  corrective  actions. 
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11.3.4.2  Design,  Analysis,  and  Evaluation.  These  elements  are  discussed  below: 

a.  Reliability  modeling  —  Preparation  of  functional  flow  and  reliability  block  diagrams  of  the 
system/equipment  down  to  the  functional  module  replacement  level.  Developing  the  math  model 
and  equations  necessary  to  enable  numerical  allocations  and  predictions 

b.  Reliability  allocation  —  Allocation  of  quantitative  system/equipment  reliability  requirements 
(mean-time-between-failures)  to  lower  assembly  units  and  to  functional  modules  based  on 
mission  and  environmental  profile  and  historical  data.  Reliability  allocation  is  a  “top-down” 
process. 

c.  Reliability/ Availability  trade-off  studies  —  Determination  of  the  optimum  design  approach 
for  considerations  of  both  reliability  and  availability  through  the  use  of  redundancy,  high 
reliability  components,  component  derating,  special  environmental  protection,  environmental 
stress  screening,  etc. 

d.  Parts  and  materials  selection,  application,  and  control  —  Establishment  of  minimum  quality 
levels  and  application  requirements  for  electrical  (e.g.,  ER  level  “P”  or  better)  and  electronic 
components  (e.g.,  JANTX  semiconductors  or  better;  MIL-M-38510  Class  “B”  microcircuits 
or  better)  and  establishment  of  derating  criteria  using  NAVSEA  TE000-AB-GTP-010  guidelines. 
Establishment  of  a  parts  identification  and  control  program  in  accordance  with  MIL-STD-965, 
procedure  1 

e.  Reliability  prediction  —  Prediction  of  the  numerical  reliability  value  (MTBF)  of  the  functional 
modules  using  MIL-STD-756,  MIL-HDBK-217,  or  other  data;  and,  using  the  module  data 
and  available  test  data,  the  prediction  of  the  numerical  reliability  value  for  the  system/equipment. 
Reliability  prediction  is  a  “bottom-up”  process. 

f.  Stress  analysis  —  Performance  of  electrical  and  thermal  stress  analyses  of  electrical  and  electronic 
components  using  NAVSEA  TE000-A8-GTP-010  as  a  guide  and  performance  of  structural 
stress  analyses  of  critical  application  mechanical  components 

g.  Variability  analysis  —  Performance  of  parameter  variability  (worst  case)  analyses  of  electrical 
and  electronic  components  using  NAVSEA  TE000-AB-GTP-010  as  a  guide  and  performance 
of  mechanical  tolerance  studies  of  functional  interface  features  of  mechanical  and 
electromechanical  devices 

h.  Sneak  circuit  analysis  —  Analysis  of  critical  circuits  to  identify  latent  paths  which  could  cause 
occurrence  of  unwanted  functions  or  could  inhibit  desired  functions 

i.  Failure  modes,  effects,  and  criticality  analysis  (FMECA)  —  Identification  of  potential  design 
weaknesses  by  determining  the  ways  an  item  may  fail,  the  cause  for  the  failure  mode,  and 
effects  and  criticality  of  the  failure  using  MIL-STD-1629 

j.  Reliability  data  bank  —  Collection  of  reliability  data,  i.e.,  equipment  operating  time,  cycle 
data,  failures,  failure  modes,  and  failure  criticality  to  assist  in  making  predictions  and  in 
determining  the  need  for  corrective  actions. 

k.  Life-cycle  effects  analysis  —  Determination  of  the  long  term  effects  of  storage,  handling, 
transportation,  maintenance,  and  repeated  testing  for  the  purpose  of  making  life-cycle  failure 
predictions  and  establishing  system/equipment  control  policies. 

11.3.4.3  Test  and  demonstration.  These  elements  are  discussed  below: 

a.  Environmental  stress  screening  —  Establishment  of  preconditioning  requirements  (e.g.,  ‘‘burn- 
in”,  temperature  cycling,  and  vibration)  for  components,  subassemblies,  and  major  functional 


11-11 


units  in  order  to  stabilize  the  equipment’s  characteristics  and  to  stimulate  early  failures  due 
to  marginal  components  or  workmanship 

b.  Reliability  growth  testing  —  Establishment  of  a  growth  testing  program,  concentrating  on 
mission-critical  failure  mode  detection,  in  order  to  provide  early  detection  and  correction  of 
reliability  problems 

c.  Reliability  demonstration  —  Demonstration  by  formal  testing  that  the  system/equipment  meets 
its  specified  reliability  requirements,  using  MIL-STD-781  or  other  equivalent  plans 

d.  Production  testing  —  Establishment  of  a  production  reliability  test  program  to  verify  that 
system/equipment  reliability  has  not  been  degraded  by  workmanship  defects,  low  quality 
components,  or  by  other  production  related  factors. 

11.3.4.4  Documentation.  The  documentation  elements  are  discussed  below: 

a.  Procurement  documentation  —  Ensuring  that  engineering  drawings  and  specifications  for 
production  of  functional  modules  and  for  designated  organizational  and  intermediate  level 
spare  parts  include  well-defined  functional  and  quality  assurance  requirements 

b.  Reliability  program  reporting  —  Provision  for  periodic  reports  on  the  status  of  each  reliability 
program  element. 

11.3.4.5  Why  Not  Depend  on  Reliability  Demonstration  Testing?  Something  more  than  definition 

is  needed: 

a.  In  order  to  demonstrate  a  5, 000-hour  mean-time-between-failure  (MTBF)  with  90  percent 
confidence  under  optimum  conditions  (0  failures),  it  would  take  over  1 1,000  hours  of  test  time 
or  458  24-hour  days. 

b.  Conclusions: 

For  high  reliability  equipment,  generally  it  is  not  practical  to  demonstrate  reliability 
statistically  by  test  prior  to  committing  the  design  to  production. 

Contractors  won’t  pay  attention  to  requirements  they  don’t  have  to  demonstrate. 

Depend  on  proven  design  disciplines  first,  then  demonstrations. 

11.3.4.6  Design  Strategies.  The  following  strategies  serve  to  satisfy  system  reliability: 

a.  Redundancy  —  Designing  one  or  more  alternate  signal  paths  into  the  system  through  addition 
of  parallel  elements 

b.  Graceful  degradation  —  Multiple  redundancy  allowing  for  automatic  switching  from  a 
malfunctioning  system  element  to  a  backup  system  element 

c.  High  reliability  design  —  Use  of  high  quality,  conservatively  derated  components.  Advantages 
over  other  strategies  include 

Reduced  initial  acquisition  costs  likely 
Lower  support  costs 
Reduced  maintenance 
Increased  availability 
Reduced  spares  requirements 
Reduced  weight  requirements. 
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11.3.4.7  Component  Derating.  Component  derating  is  discussed  below: 

a.  Component  failure  rates  and  therefore  equipment  MTBF  are  directly  related  to  stress. 

b.  In  order  to  develop  a  reliable  design,  the  designer  must  identify  and  control  component  stress 
levels. 

c.  Derating  is  increasing  the  ratio  (margin  of  safety)  between  part  design  limits  and  the  applied 
stresses  (i.e.  electrical,  thermal). 

d.  NAVSEA  TE000-AB-6TP-010,  Parts  Application  and  Reliability  Information  Manual  for  Navy 
Electronic  Equipment,  provides  derating  guidelines  for 

Microcircuits 

Semiconductors 

Resistors 

Capacitors 

Connectors 

Switches 

Relays 

Transformers,  inductors,  and  coils. 

e.  Derating  provides  added  protection  from  part  variances,  decreased  part  degradation  rate,  and 
increased  expected  life. 

11.3.4.8  Component  Quality  Levels.  The  quality  levels  for  components  are  listed  below: 

a.  The  use  of  properly  screened  and  qualified  components  which  are  conservatively  derated  in 
their  circuit  application  is  the  best  assurance  of  reliable  electronic  hardware. 

b.  Component  quality  levels 

Active  and  passive  electrical  components  (e.g.,  relays,  coils,  connectors, 


resistors,  capacitors); 

established  reliability  (ER)  1 

failure 

rates 

ER  failure  rate 

“L” 

=  2.0% 

per 

1,000 

hrs 

ER  failure  rate 

“M” 

=  1.0% 

per 

1,000 

hrs 

*ER  failure  rate 

»«p>i 

=  0.1% 

per 

1,000 

hrs 

ER  failure  rate 

“R” 

=  0.01% 

per 

1.000 

hrs 

ER  failure  rate 

“S” 

=  0.001% 

per 

1,000 

hrs 

ER  failure  rate 

ujt » 

=  0.0001% 

per 

1,000 

hrs 

Discrete  semiconductors  (e.g.  MIL-S-19500  transistors,  diodes);  failure  rates  compared 
to  equivalent  commercial  devices 


JAN 

Failure  rate 

=  0.2 

‘JANTX 

Failure  rate 

=  0.04 

JANTXV 

Failure  rate 

=  0.02 

JANS 

Failure  rate 

=  0.005 

Where  JANTX  devices  are  unavailable,  approved  substitutes  to  be  screened  to  MIL 
STD-750  JANTX  requirements. 

Microcircuits  (MIL-M-38510);  failure  rates  compared  to  equivalent  commercial  devices 

Class  C  Failure  Rate  =  0. 1 

*Class  B  Failure  Rate  =  0.01 

Class  S  Failure  Rate  =  0.006 


Where  ciass  “B”  devices  are  unavailable,  approved  substitutes  to  be  screened  to  MIL- 
STD-883,  Method  5004,  class  “B”  requirements. 

•NAVSEASYSCOM  design  requirement  (specified  level  or  better) 

c.  The  case  for  using  quality  components  has  two  main  points: 

Components  represent  approximately  25  percent  of  the  equipment  cost  when  commercial 
grade  electronic  components  are  used. 

If  ER  level  “P”  active  and  passive  electrical  components,  MIL-S-19500  JANTX  discrete 
semiconductors,  and  MIL-M-38510  class  “B”  microcircuits  are  used,  the  parts  cost  is 
increased  by  100  percent  and  the  equipment  cost  is  increased  by  25  percent.  However, 
the  predicted  equipment  MTBF  is  increased  14  to  20  times  over  that  of  equipment 
constructed  with  commercial  grade  components! 

11.3.4.9  Environmental  Stress  Screening  (ESS).  ESS  is  discussed  in  the  items  below. 

a.  Environmental  stress  screening  is  the  application  of  electrical  and  environmental  (temperature, 
vibration)  stress  to  precipitate  latent  defects  (components,  workmanship)  at  levels  of  assembly 
where  defect  correction  is  most  cost  effective. 

b.  ESS  is  not  a  test,  it  is  a  manufacturing  process. 

c.  There  is  no  relationship  between  the  environmental  levels  used  for  ESS  and  those  used  for 
qualification  testing.  ESS  levels,  typically  temperature,  often  are  higher  than  qualification  levels. 

d.  Properly  designed  ESS  will  not  damage  good  hardware,  nor  appreciably  reduce  its  useful  life. 

e.  Environmental  stress  screening  plan  guidelines 

Thermal  cycling  (applies  100  percent  to  components  and  100  percent  to  modules,  units, 
or  assemblies) 

Cycling  between  —  40°C  (-40°F)  and  90°C  (194°F)  with  a  5°C/minute 
minimum  rate  of  change 

10  cycles  minimum  (20  to  30  desirable)  with  sufficient  dwell  time  (10  minutes  for 
modules)  to  ensure  stability 

Wherever  possible,  power  applied  to  modules,  etc.  During  heating  portion  of  cycle 

Performance  measurements  on  modules,  etc.  to  be  made  at  operating  temperature 
extremes  during  cycling  on  systematic  basis. 

Following  cycling,  components  (i.e.  discrete  semiconductors,  integrated  circuits)  to 
be  subjected  to: 

Electrical  tests  (i.e.  static,  dynamic,  functional)  at  25°C  and  125°C 

Particle  impact  noise  tests  on  hybrids  and  unglassivated  semiconductors  having 
cavities 

Hermeticity  test  recommended  for  sealed  devices. 

Vibration  (applies  100  percent  to  modules,  units,  or  assemblies) 

Random  vibration  in  two  axes  for  10  minutes  minimum,  per  axis 

Acceleration  spectrum  of  0.04g;/Hz  from  20  to  2,000  Hz  with  3  dB/octave  roll  off 
from  80  to  20  Hz  and  from  350  to  2,000  Hz 
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Performance  measurements  to  be  made  before  and  after  cycling 

ESS  plan  should  be  adjusted  as  product  matures  during  full-scale  phase  development  and 
during  production  (based  on  process  results). 

ESS  requirements,  when  fully  mature,  should  be  included  on  engineering  drawings  for 
units,  modules,  and  assemblies  which  may  be  reprocured  as  spares. 

Destructive  physical  analysis  recommended  in  conjunction  with  component  lot  acceptance 
inspection. 

Refer  to  NAVMAT  P-9492,  NAVEA  Notice  3900,  IES  guidelines,  and  proposed 
NAVSPAWAR  and  RADC  MIL-STDs  for  additional  ESS  guidance. 

11.3.5  Maintainability  Assurance  Program  Elements 

11.3.5.1  Program  Planning,  Monitoring,  and  Control.  These  elements  are  discussed  below: 

a.  Program  planning  —  Definition  of  the  maintainability  program  (M1L-STD-470)  in  a  plan  which 
identifies  and  describes  all  program  monitoring,  control,  design,  analysis,  evaluation, 
demonstration,  and  documentation  elements 

b.  Subcontractor/Supplier  monitoring  and  control  —  Monitoring  by  the  prime  contractor  of  all 
subcontractor/supplier  maintainability  program  efforts  and  ensuring  compliance  with 
maintainability  program  efforts  and  ensuring  compliance  with  the  overall  program  requirements 

c.  Program/Design  reviews  —  Review  of  the  maintainability  program  progress  at  specified  points 
in  time  and  conduct  of,  as  a  minimum,  preliminary  and  critical  design  reviews  per  MIL-STD-1521 

d.  Maintainability  deficiency  reporting  —  Establishment  of  a  closed-loop  deficiency  reporting 
system  providing  for  the  analysis  of  a  correction  of  maintainability  problems. 


11.3.5.2  Design,  Analysis,  and  Evaluation.  These  elements  are  discussed  below: 

a.  Maintainability  modeling  —  Preparation,  in  conjunction  with  the  maintainability  analysis,  of 
maintainability  block  diagrams  down  to  the  major  assembly  or  module  replacement  level. 
Development  of  the  math  model  and  equations  necessary  to  enable  numerical  allocations  and 
predictions 

b.  Maintainability  allocation  —  Allocation,  in  conjunction  with  the  maintainability  analysis,  of 
quantitative  system  equipment  maintainability  requirements  (mean-time-to-repair)  to  the  major 
assembly  or  to  the  module  replacement  level.  A  top-down  process 

c.  Maintainability  analysis  —  Translation  of  various  system/equipment  analysis  and  Navy  operating 
constraints  data  into  detailed  quantitative  and  qualitative  maintainability  requirements  and  into 
the  detailed  maintenance  plan.  Such  analysis  data  includes:  operational  and  support  requirements; 
environmental  conditions;  overall  quantitative  maintainability  requirements;  projected 
facilities/equipment /skills  availability. 

d.  Maintainability  design  trade-off  studies  —  Determination,  in  conjunction  with  the  maintainabilitv 
analysis  and  whenever  design  trade-offs  are  performed  for  other  reasons,  of  the  effect  of  the 
design  approach  on  the  maintainability  aspects  of  the  system  /equipment 


e.  Maintainability  design  criteria  —  Establishment,  in  conjunction  with  the  maintainability  analysis, 
of  those  maintainability  design  criteria  that  are  to  be  considered  for  incorporation  into  the  design 
including:  accessibility;  work  space;  work  clearance;  component  interchangeability  and 
standardization,  limiting  the  numbers  and  varieties  of  tools  and  support  equipment;  use  of 
maintenance-free  components;  adequate  tolerance  and  wear  factors;  failure  design;  rapid  fault 
detection  and  localization;  ease  of  adjustment  and  calibration;  limiting  personnel  numbers  and 
skills  requirements;  application  of  human  engineering  principles;  avoiding  the  potential  for 
maintenance  errors;  etc. 

f.  Maintainability  prediction  —  Prediction  of  the  maintainability  value  (MITR)  of  the 
system/equipment  using  MIL-HDBK-472  or  other  acceptable  techniques.  Such  predictions  should 
reflect  applicable  experience  with  similar  systems/equipment.  A  bottom-up  process 

g.  Maintainability  data  bank  —  Establishment  of  a  maintainability  data  bank,  to  be  integrated 
with  the  reliability  data  bank,  to  assist  in  making  predictions,  and  to  assist  in  evaluating  the 
demonstration  results 

h.  Maintainability  demonstration  —  Demonstration  of  the  achievement  of  the  qualitative  and 
quantitative  (MITR)  maintainability  requirements  for  the  system/equipment.  To  be  accomplished 
in  accordance  with  MIL-STD-471 

i.  Maintainability  program  reporting  —  Provision  for  periodic  reports  on  the  status  of  each 
maintainability  program  element. 


11.4  SYSTEM  SAFETY  ASSURANCE 


11.4.1  Definitions 


11.4.1.1  System  Safety  Assurance.  This  is  the  continuing  analysis  and  monitoring  of  system/ equipment 
design,  operation,  and  maintenance  to  assure  that  the  optimum  degree  of  safety  is  attained  within  the 
constraints  of  operational  effectiveness,  time,  and  cost. 

11.4.1.2  Safety.  This  is  freedom  from  conditions  that  can  cause  death,  injury,  occupational  illness, 
or  damage  to  or  loss  of  equipment  or  property. 

11.4.1.3  System  Safety.  This  is  the  application  of  engineering  and  management  techniques  to  optimize 
safety  within  the  constraints  of  operational  effectiveness,  time,  and  cost  throughout  all  life-cycle  phases. 

11.4.1.4  Mishap.  This  is  an  unplanned  event  or  series  of  events  that  results  in  death,  injury,  occupational 
illness,  or  damage  to  or  loss  of  equipment  or  property 

11.4.1.5  Hazard.  Condition  prerequisite  to  a  mishap, 
a.  Hazard  categories: 

I  (catastrophic)  —  Death  or  system  loss 

II  (critical)  —  Severe  injury,  severe  occupational  illness,  or  major  system  damage 

III  (marginal)  —  Minor  injury,  minor  occupational  illness,  or  minor  system  damage 

IV  (negligible)  —  Less  than  minor  injury,  occupational  illness,  or  system  damage 
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b.  Hazard  probabilities:  1  For  individual  item 

2  For  Fleet  inventory 

A  (frequent)  —  1  Likely  to  occur  frequently 

2  Continuously  experienced 

B  (probable)  —  1  Will  occur  several  times  in  life  of  item 

2  Will  occur  frequently 

C  (occasional)  —  1  Likely  to  occur  in  life  of  item 

2  Will  occur  several  times 

D  (remote)  —  1  Unlikely,  but  possible  to  occur  in  life  of  item 

2  Unlikely,  but  can  reasonably  be  expected  to  occur 

E  (improbable)  —  1  So  unlikely,  it  can  be  assumed  it  may  not  occur 

2  Unlikely  to  occur,  but  possible 

11.4.2  System  Safety  Program  Policy  (NAVMATINST  5100.6A) 

11.4.2.1  Policy.  System  safety  program  to  be  established  in  connection  with  all  acquisitions,  to  ensure: 
Regard  for  system  safety  is  a  fundamental  element  of  the  acquisition  process 

Personnel  will  not  be  unnecessarily  exposed  to  injury  or  health  hazards 
Equipment  and  property  will  not  be  unnecessarily  subjected  to  damage. 

11.4.2.2  Technical  Requirements.  System  safety  program  based  on  MJL-STD-882: 

Program  plan  to  be  prepared 

Hazard  analyses  to  be  conducted  and  hazards  identified  and  categorized 
Action  taken  to  eliminate  or  control  hazards,  preferably  by  d?.-ign 

Where  normal  testing  does  not  demonstrate  safe  operation,  special  safety  tests  to  be  conducted 
Document  program  efforts. 

11.4.3  System  Safety  Assurance  Program  Elements 


11.4.3.1  Program  Planning  and  Control.  These  elements  are  discussed  below: 

a.  Program  planning  —  Definition  of  the  system  safety  program  (M1L-STD-882)  in  a  plan  which 
identifies  and  describes  all  program  monitoring,  control,  analysis,  evaluation,  testing,  and 
documentation  elements 

b.  Subcontractor  monitoring  and  control  —  Monitoring  by  the  prime  contractor  of  all  subcontractor 
system  safety  program  efforts  and  ensuring  compliance  with  the  overall  program  requirements 

c.  Program/Design  reviews  —  Review  of  the  system  safety  program  progress  at  specified  points 
in  time  and  conduct  of,  as  a  minimum,  preliminary  and  critical  design  reviews  per  M1L-STD-I521 

d.  Failure  reporting  —  Establishment  of  a  closed-loop  failure  reporting  system  providing  for  the 
analysis  of  and  correction  of  safety-related  failures. 
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11.4.3.2  Analysis  and  Evaluation.  These  elements  are  discussed  below: 

a.  Preliminary  hazard  analysis  —  Assessment  of  the  initial  risk  of  a  system/equipment  or  concept 
in  order  to  identify  safety  critical  areas,  evaluate  hazards,  and  identify  the  safety  design  criteria 
to  be  used.  The  analysis  considers  the  following  for  identification  of  hazards: 

Hazardous  components  (e.g.,  energy  sources,  fuels,  propellants,  explosives,  pressure  systems) 

Safety-related  interface  considerations  (e.g.,  materials  compatibility,  electromagnetic  radiation 
interference,  fire/ explosion  susceptibility,  and  propagation  potential) 

Environmental  constraints,  including  the  normal  operating  environment  (e.g.,  vibration,  shock, 
temperature,  noise  or  health  hazards,  fire  electrostatic  discharge,  lightning.  X-ray, 
electromagnetic,  and  nuclear  and  laser  radiation) 

Operating,  test  maintenance,  and  emergency  procedures  (e.g.,  human  error  possibilities, 
environmental  effects  on  human  performance,  life-support  requirements  in  manned  systems, 
crash  survival/rescue) 

Facilities,  support  equipment,  training  (e.g.  provisions  for  storage,  assembly,  testing  or  training 
regarding  hazardous  systems  or  where  toxic,  flammable,  explosive,  corrosive,  or  cryogenic 
materials  are  used  in  these  activities) 

Safety-related  equipment,  safeguards,  and  alternate  design  approaches  (e.g.,  interlocks, 
redundancy,  fail-safe  design  features,  personal  protective  gear,  fire  suppression  systems). 

b.  Subsystem  hazard  analysis  —  Identification  of  hazards  associated  with  component  failures  within 
a  given  subsystem.  Performed  when  detailed  design  is  completed.  Analysis  techniques  used 
include: 

Fault  hazard  analysis  —  An  inductive  method  of  analysis  involving  a  detailed  investigation 
of  subsystem  component  hazard  modes,  causes  of  hazards,  and  effects  on  the  subsystem. 
The  analysis  may  be  qualitative  or  expanded  to  a  quantitative  one. 

Fault  tree  analysis  —  A  deductive  analysis  of  all  events,  faults,  and  occurrences  that  could 
cause  or  contribute  to  the  occurrence  of  undesired  events.  The  analysis  may  be  qualitative 
or  quantitative. 

Sneak  circuit  analysis  —  Attempts  to  identify  latent  (sneak)  circuits  and  conditions  that  inhibit 
desired  functions  or  cause  undesired  functions  to  occur  without  an  accompanying  component 
failure. 

c.  System  hazard  analysis  —  Performed  on  subsystem  interfaces  to  determine  the  hazard  problem 
areas  of  the  total  system.  The  fault  hazard,  fault  tree,  and  sneak  circuit  analysis  techniques 
are  used.  The  analysis  should  consider  the  following  subsystems  relationships: 

Compliance  with  safety  criteria 

Possible  independent,  dependent,  or  simultaneous  failures 

Safety  degradation  of  one  subsystem  under  normal  operating  conditions  of  another. 

d.  Operating  and  support  hazard  analysis  —  Identification  of  potential  hazards  during  production, 
installation,  maintenance,  testing,  modification,  transportation,  storage,  operation,  training, 
or  during  other  phases  of  use  or  disposal.  Results  of  analysis  will  be  used  to  control  hazards 
and  to  determine  appropriate  safety  requirements  for  personnel,  procedures,  and  equipment, 
including: 


Identifying  times  of  high  hazards  and  actions  required  to  minimize  risks 
Design  changes  necessary  to  eliminate  or  control  hazards 

Identifying  requirements  for  safety  devices  and  equipment  and  required  procedures  for  ensuring 
their  proper  operation 

Warnings,  cautions,  and  emergency  procedures  for  operation  and  maintenance 
Special  procedures  for  operation,  handling,  storage,  transportation,  and  modification. 

e.  Safety  testing  and  demonstrations  —  Performance  of  tests  or  demonstrations  on  safety  critical 
equipment  and  procedures  to  determine  the  hazard  severity  or  to  establish  the  margin  of  safety 
of  the  design 

f.  System  safety  program  report  —  Provision  for  periodic  reports  on  the  status  of  each  system 
safety  program  element. 


11.5  HUMAN  FACTORS 


11.5.1  Definitions 

Human  factors  engineering  (i.e.  human  engineering)  is  the  planned  integration  of  people  and 
equipment  into  an  effective  operating  system 

11.5.2  Human  Factors  Policy  (NAVMATINST  3900.9) 

11.5.2.1  Objectives.  The  human  factors  objectives  are  as  follows: 

a.  To  ensure  the  development  of  optimum  man-machine  systems 

b.  To  ensure  that  human  tasks  designed  into  systems  are  consistent  with  manpower  capabilities 

c.  To  ensure  that  training  programs  and  equipment  are  developed 

d.  To  reduce  manpower  life-cycle  costs  by  reducing  manpower,  skill  level,  and  training  requirements 

e.  To  ensure  that  the  physiological  environment  of  a  system  is  consistent  with  its  performance  goals. 

11.5.2.2  Policy.  The  human  elements  of  systems  are  to  undergo  the  same  development,  test,  and 
evaluation  as  equipment  elements. 

11.5.2.3  Technical  requirements.  Requirements  of  MIL-H-46855,  Human  Engineering  Requirements 
for  Military  Systems,  Equipment  and  Facilities,  are  to  apply  during  system  development,  test,  and 
evaluation. 

11.5.3  Human  Engineering  Program  Elements 

11.5.3.1  Analysis.  This  element  is  discussed  below: 

a.  Definition  and  analysis  of  the  functions  to  be  performed  by  the  man-machine  system  in  order 
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to  specify  the  performance  requirements  for  system  operation,  maintenance,  and  control  and 
to  allocate  system  functions  to  automatic  operation/maintenance,  manual  operation/ 
maintenance,  or  a  combination 

b.  Application  of  human  engineering  principles  and  criteria  in  the  identification  and  selection  of 
equipment  to  be  operated/maintained  by  personnel.  The  requirements  of  MIL-STD-1472,  Human 
Engineering  Design  Criteria  for  Military  Systems,  Equipment  and  Facilities,  apply. 

c.  Analysis  of  tasks  to  identify  information,  decisionmaking,  activity,  facilities,  etc.  are  required. 

11.5.3.2  Equipment/Facilities  Design  Input.  This  input  is  discussed  below: 

a.  Conducting  experiments,  tests,  and  studies  to  resolve  human  engineering  and  life  support 
problems 

b.  Human  engineering  input,  based  on  analyses  conducted,  to  detail  equipment  design  features 
as  reflected  by  the  engineering  drawings 

c.  Human  engineering  input,  based  on  analyses  conducted,  to  the  work  environment,  operator/crew 
stations,  and  facilities  designs. 

11.5.3.3  Test  and  Evaluation.  These  elements  are  discussed  below: 

a.  Planning  of  tests  to  verify  that  the  system/equipment  can  be  operated,  maintained,  supported, 
and  controlled  by  user  personnel  in  the  intended  environment 

b.  Conducting  of  tests  and  identification  of  discrepancies 

c.  Analysis  of  failures  and  identification  of  solutions. 

11.5.3.4  Coordination.  This  is  coordination  of  human  engineering  activity  with  reliability,  maintain¬ 
ability,  system  safety,  survivability/vulnerability,  and  integrated  logistic  support  functions. 


11.6  QUALITY  ASSURANCE 

11.6.1  Definitions 

11.6.1.1  Quality  Assurance.  This  is  the  planned  and  systematic  technical  direction  and  surveillance 
of  a  producer’s  design,  materials,  components  controls,  manufacturing  processes,  and  inspection  and 
test  practices  to  assure  the  deliver}'  of  systems/equipment  which  will  satisfy  the  user’s  requirements. 

11.6.1.2  Quality.  This  is  the  composite  of  all  the  attributes  or  characteristics,  including  performance, 
of  a  product. 

11.6.1.3  Inherent  Quality.  This  is  the  presence  in  the  design  of  those  attributes  (e.g.  performance, 
reliability,  survivability,  maintainability,  system  safety,  human  factors)  necessary  to  satisfy  the  user’s 
requirements,  i.e.  design  quality. 

11.6.1.4  Manufacturing  Quality.  This  is  the  conformance  of  a  manufactured  product  to  its  required 
drawings  and  specifications. 

11.6.1.5  Achieved  Quality.  The  ability  of  a  manufactured  product  to  satisfy  the  user’s  requirements, 
i.e.  overall  product  quality. 


Achieved  Quality  =  Inherent  Quality  +  Manufacturing  Quality 

or 

Overall  Product  Quality  =  Design  Quality  4-  Manufacturing  Quality. 

11.6.2  Quality  Assurance  Program  Policy  (NAVMATINST  4855.1) 

11.6.2.1  Policy.  The  policy  is  discussed  below: 

a.  Quality  is  to  be  a  major  factor  in  system  planning,  engineering,  and  management. 

b.  Inherent  quality  is  established  by  the  design. 

c.  Assurance  of  achieved  quality  requires  a  planned  program. 

d.  Measurement  of  achieved  quality  must  be  continuous. 

11.6.2.2  Program  Requirements.  The  requirements  are  discussed  below: 

a.  Quality  requirements  to  be  included  in  contracts  and  consideration  given  to  quality  during  source 
selection 

b.  Contractors’  quality  programs  to  be  evaluated  and  audited 

c.  Engineering  specifications  to  include  quality  assurance  provisions 

d.  Quality  requirements  to  be  based  on  operating  environment 

e.  Contractor  developed  test  and  inspection  equipment  programs  to  be  assessed 

f.  Quality  assurance  requirements  to  be  established  in  connection  with  packaging,  handling, 
shipment,  and  storage 

g.  Quality  assurance  requirements  to  be  established  in  connection  with  maintenance  operations 

h.  Quality  concepts  to  be  included  in  MIL-STDs,  MIL-SPECS,  and  QPLs. 

11.6.3  Quality  Assurance  Program  Elements 

11.6.3.1  Quality  Assurance  Program  Plan.  This  is  the  plan  for  assuring  the  quality  of  the  design, 
design  documentation,  and  fabricated/assembled  hardware  and  associated  computer  software 

11.6.3.2  Hardware  Quality  Assurance  Program  Provisions.  These  elements  are  discussed  below: 
a.  Management 

Responsibility — Identifies  the  organization  and  individual  responsible  for  program 
implementation,  management,  and  control.  Defines  responsibilities  and  authority.  Identifies 
chain-of-authority  for  reporting  purposes. 

Work  instructions 

Development  hardware.  Establishes  requirement  that  special  or  nonroutine  procedures  (e.g. 
manufacturing,  assembly,  calibration,  alignment,  testing)  be  described  by  written  work 
instructions 

Fabrication/assembly  of  Fleet  service  systems/equipment.  Establishes  requirement  that 


all  manufacturing  procedures  to  be  described  by  detailed,  written  work  instructions 
maintained  under  strict  configuration  control 

b.  Design  control 

Product  baseline — Provides  an  exact  definition  (listing)  of  applicable  drawings, 
specifications,  and  approved  changes  to  which  hardware  will  be  fabricated,  inspected,  and 
tested. 

Change  control — Establishes  the  system  for  documenting,  controlling,  and  accounting  for 
engineering  changes,  deviations,  and  waivers,  from  both  the  administrative  approval  and 
hardware  implementation  standpoints. 

c.  Procurement  actions  (i.e.  subcontracts,  purchase  orders) 

Procurement  requirements — Includes  technical  and  quality  assurance  requirements  such 
as:  MIL-Q-9858  quality  program/MIL-I-45208  inspection  system;  piece  part/first 
article/unit  acceptance/lot  acceptance  inspection  and/or  test;  ESS;  change  control; 
workmanship;  soldering;  government  inspection/acceptance  authority 

Source  inspection — Expresses  ordering  activity’s  intention  to  inspect  units  of  product  at 
source 

Parts/Materials — Includes  requirement  that  purchased  components  or  fabricated  items  be 
inspected/tested  by  subcontractor  upon  receipt.  Requires  certifications  for  critical  raw 
materials. 

d.  Material  control 

Identification — Requires  identification,  segregation,  and  control  of  incoming  material 
awaiting  inspection,  material  having  completed  inspection  and  found  acceptable,  and 
nonconforming  material 

Handling  control— Requires  proper  handling  of  material  during  processing 

Storage  control — Establishes  requirements  for  preservation  and  packaging  of  completed 
material 

Shipment  control — Establishes  requirements  for  proper  preparation  for  shipment. 

e.  Manufacture/ Assembly 

Process  control — Establishes  evaluations,  controls,  and  inspections  at  appropriate  points 
in  the  manufacturing  process  to  ensure  continuous  control  over  quality  of  produced  products 

Special  processes — Establishes  requirements  for  methods  and  facilities  used  in  connection 
with  soldering,  brazing,  welding,  bonding,  encapsulating,  plating,  anodizing,  heat  treating, 
nondestructive  testing,  ESS,  etc.  Requires  certification  of  personnel  performing  special 
processes. 

Workmanship — Establishes  requirements  for  general  workmanship  (MIL-STD-454, 
Requirement  9)  practices  and  any  special  product  workmanship  provisions. 

f.  Acceptance  inspection  and  testing 

First  article/preproduction  sample — Provides  confidence  that  producer  is  capable  of 
manufacturing  items  that  meet  the  full  performance  and  environmental  requirements  of 
the  drawings  and  specifications 

Unit  acceptance — Establishes  physical  inspection  and  performance  test  requirements  for 
conditional  acceptance  of  individual  units  of  lot.  100  percent  testing  of  functional  units 


and  critical  physical  features.  Large  lots  of  homogeneous,  noncritical  units  are  inspected 
on  a  statistical  sample  basis. 

Periodic  production  lot  sample — Ensures  that  randomly  selected  production  sample  items 
meet  the  full  performance  and  environmental  requirements  of  the  drawings  and 
specifications.  Determines  acceptability  of  lot. 

g.  Corrective  action 

Noncomforming  material — Establishes  procedure  for  rework,  repair,  scrap,  return  to 
vendor,  or  other  disposition  (e.g.  waiver  action)  of  discrepant  material 

Action  to  prevent  recurrence — Establishes  technical  and/or  administrative  action  necessary 
to  prevent  recurrence  of  the  discrepancy.  Includes  monitoring  of  effectiveness  of  corrective 
action  following  implementation. 

h.  Measuring  and  testing  equipment  control — Ensures  all  measuring  and  test  equipment  used  for 
material  acceptance  or  evaluation  purposes  is  calibrated  before  use  and  is  subject  to  a  calibration 
recall  program 

i.  Quality  information 

Records — Establishes  requirements  to  maintain  records  concerning  inspections/tests  of  both 
conforming  and  nonconforming  units  of  product 

Quality  cost  data — Identifies  the  cost  of  both  the  prevention  and  correction  of 
nonconforming  supplies. 

11.6.4  Why  Inspect  at  the  Piece  Part  Level? 

Review  the  Burroughs  Corporation  experience  concerning  the  comparative  costs  of  locating  and 
replacing  a  defective  semiconductor  at  various  assembly  levels  in  an  item  of  equipment — J.  Zeccardi 
(Vice  Pres,  for  Quality/Service) 

Defect  Found  During:  Cost  to  Locate/Replace 


Inspection  by  supplier 

$.03 

Incoming  inspection  at  Burroughs 

$.30 

Subassembly  (circuit  board)  test 

$3.00 

Assembly  test 

$30.00 

Equipment  test 

$300.00 

Field  test 

$3,000.00 

11.7  CONFIGURATION  MANAGEMENT 

11.7.1  Definitions 

11.7.1.1  Configuration  Management.  This  is  the  technical  and  administrative  direction  and  surveillance 
of  the  functional  and  physical  characteristics  of  system/equipment  hardware  or  computer  software. 

11.7.1.2  Configuration  Identification.  This  is  the  identification  and  documentation  of  the  functional 
and  physical  characteristics  of  hardware/software  with  engineering  drawings,  parts  lists,  specifications,  etc. 
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11.7.1  J  Configuration  Control.  This  is  the  technical  analysis  and  control  of  changes  to  these  functional 
and  physical  characteristics  as  documented  by  engineering  change  proposals  (ECPs),  deviations,  and 
waivers. 

11.7.1.4  Configuration  Status  Accounting.  This  is  the  recording  of  the  approved  configuration 
identification  in  functional,  allocated,  or  product  baselines,  and  the  reporting  of  the  status  of  change 
(ECP)  processing  and  implementation. 

11.7.1.5  Configuration  Audits.  These  are  the  formal  verifications  during  full-scale  development, 
through  physical  testing  (functional  configuration  audit)  and  examination  (physical  configuration  audit), 
that  the  hardware  and  its  related  configuration  identification  meet  contractual  requirements  and  program 
needs. 

11.7.1.6  Engineering  Change  Proposal  (ECP).  The  ECP  includes  both  a  proposed  engineering  change 
and  the  documentation  by  which  the  change  is  described  for  purposes  of  incorporation  into  the  affected 
drawings,  etc. 

11.7.1.7  Deviation.  This  is  a  specific  written  authorization,  granted  prior  to  the  manufacture  of 
an  item,  to  depart  from  a  particular  design  or  performance  requirement. 

11.7.1.8  Waiver.  This  is  written  authorization  to  accept  an  item  which  during  production  or  after 
having  been  submitted  for  inspection  is  found  to  depart  from  specified  requirements. 

11.7.2  Configuration  Management  Policy  (NAVMATINST  4130.1) 

The  requirements  of  NAVMATINST  4130.1  are  described  by  paragraph  3,  following. 

11.7.3  Configuration  Management  Program  Elements  (NAVMATINST  4130.1) 

11.7.3.1  Configuration  Identification.  These  elements  are  discussed  below: 

a.  Functional  baseline  (basis  for  validation  phase) 

Type  “A”  system  specification 

Level  1  system/equipment  drawings 

b.  Allocated  baseline  (basis  for  full-scale  development) 

Type  “A”  system  specification 

Type  “B”  development  specifications 
Level  Vi  system  drawings 
Interface  control  drawings 

c.  Product  baseline  (basis  for  production) 

Type  “C”  product,  type  “D”  process,  type  “E”  material  specifications 
Level  lA  system/equipment  drawings 
Level  3  spare  parts  drawings 
Installation  control  drawings. 

11.7.3.2  Configuration  Control  (DOD-STD-480  describes).  These  elements  are  discussed  below: 
Engineering  change  proposals  (Class  I,  II  ECPs) 
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Deviations  (critical,  major,  minor) 

O  Waivers  (critical,  major,  minor) 

' '  Material  review  board  actions. 

11.7.3.3  Configuration  Status  Accounting  (M1L-STD-482  describes).  These  elements  are  discussed 
below: 

Baseline  status 

ECP,  deviation,  waiver  approval  status 

ECP,  deviation,  waiver  contract  implementation  status 

“As  built”  configuration  records 

Fleet  system/equipment  configuration  records. 

11.7.3.4  Configuration  Audits  (MIL-STD-1521  provides  guidelines).  These  elements  are  discussed 
below: 

Functional  configuration  audit  (FCA). 

Physical  configuration  audit  (PC A). 


11.8  DESIGN  ASSURANCE 


11.8.1  Definition 

Design  assurance  is  the  technical  direction  and  monitoring  of  the  system/equipment  design  process 
,-V-  to  assure  that  the  design  appears  to  be  reliable,  maintainable,  safe,  and  producible,  and  that  the 
engineering  drawings  and  specifications  are  complete  in  their  disclosure  of  the  design  and  contain 
appropriate  product  assurance  requirements. 


11.8.2  Design  Assurance  Policy 

Other  than  that  policy  dealing  with  design  reviews  there  is  no  Navy  policy  dealing  with  the  subject 
of  design  assurance,  although  the  DSARC/NSARC  reviews  include  that  concern. 


11.8.3  Design  Assurance  Program  Elements 


11.8.3.1  Development  Specifications  (For  full-scale  development).  These  specifications  are  discussed 
below: 

a.  System/equipment  full-scale  phase  development  must  not  begin  until  the  following  are  defined 
in  a  MIL-S-83490  type  “A”  system  specification: 

System/equipment  performance  requirements 
Detailed  mission  profile 
Environmental  requirements 

Qualitative  and  quantitative  reliability  and  maintainability  requirements. 

b.  Type  “B”  development  specifications  appropriate  for  large/complex  system  development 
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11.8.3.2  Design  Documentation  Requirements  (for  full-scale  development).  System/equipment  full- 
scale  phase  development  must  not  begin  until  the  design  documentation  requirements,  based  on  the 
anticipated  production  procurement  and  logistic  support  plans,  are  established.  These  requirements 
should  include  the  following: 

DoD-D-1000,  level  3  drawings  are  required  for  competitively  awarded  production  and  all  spare  parts. 
Level  2  drawings  appropriate  for  one-time-only  production  by  original  developer. 

MIL-S-83490  type  “C”  product,  type  “D”  process,  and  type  “E”  material  specifications,  as  appropriate 
Functional  test  requirements  for  testable  subassemblies  intended  to  be  reprocured  as  spare  parts 

Documentation  of  special  (i.e. ,  not  covered  by  military  or  other  recognized  standards)  functional 
components 

Schematics  and  wiring  diagrams 

Documentation  of  interconnect  harnesses  and  cables  using  grids 

Printed  wiring  boards  and  flexible  circuits  designed  to  MIL-STD-275,  designed  to  be  producible  to 
MIL-P-55110  and  solderable  to  WS-6536 

Prohibiting  use  of  company  material  and  process  standards 

Use  of  specification,  altered,  and  selected  item  control  drawings.  Use  of  source  control  drawings  should 
require  specific  government  approval 

Parts  and  index  lists.  Computer  generation  of  lists 
Use  of  mono-  and  multi-detail  drawings 
Component  identification 
Use  of  fractions  to  be  avoided 
Use  of  metrics 

Use  of  Navy  drawing  formats,  numbers,  and  FSCMs 
Use  of  DoD-STD-100  and  ANSI  Y14.5 

Soldering  requirements  (WS-6536  applies)  to  be  included  on  drawings 
Workmanship  requirements  (MIL-STD-454,  Req  9  applies)  to  be  included  on  drawings 
Classification  of  characteristics  (DoD-STD-2101  applies)  for  items  to  be  reprocured  as  spares 
Use  of  computer-aided  design  techniques 

Use  of  proprietary  components,  designs,  and  data  to  require  advance,  written  approval. 

11.8.3.3  Design  Reviews.  Design  reviews,  conducted  following  the  guidelines  of  MIL-STD-1521, 
to  be  performed: 

a.  System  requirements  review  (SRR) — Evaluates  system  engineering  activity  and  output  regarding 
mission  and  requirements  analysis,  system/cost  effectiveness,  hardware/computer  software  trade¬ 
off  analyses,  reliability/maintainability  trade-off  analyses,  integrated  logistic  support  analysis, 
etc.  Conducted  near  end  of  conceptual  phase  or  early  in  validation  phase. 

b.  System  design  review  (SDR)— Evaluates  the  adequacy  of  the  total  system  requirements,  including 
the  Type  A  system  specification,  for  suitability  for  proceeding  into  full-scale  development. 
Conducted  near  the  end  of  the  validation  phase  or  as  an  initial  step  in  the  full-scale  development 
phase. 


c.  Preliminary  design  review  (PDR) — Evaluates  the  basic  design  approach,  including  trade-off 
studies,  preliminary  mechanical/electrical  design,  reliability/maintainability  apportionment, 
producibility,  use  of  off-the-shelf  designs/equipment,  safety  and  human  factors  analyses,  logistic 
supportability,  etc.  Conducted  early  in  full-scale  development  phase  prior  to  initiating  detail 
design  effort. 

d.  Critical  design  review  (CDR)— Evaluates  the  detail  design  solutions  to  satisfy  the  performance, 
mission  profile,  environmental,  reliability,  maintainability,  safety,  human  factors,  and  logistic 
supportability  requirements.  Conducted  during  the  full-scale  development  phase  prior  to  release 
of  the  detailed  design  to  fabrication. 

11.8.3.4  Design  Documentation  Review.  Prior  to  release  of  the  engineering  drawings  and 
specifications  to  production,  at  the  completion  of  full-scale  development,  an  independent  review  must 
be  conducted  to  verify: 

a.  Detailed  design  approach  appears  to  be  reasonable,  reliable,  and  maintainable 

b.  Mil-Spec  components  and  materials  selected  where  available.  Special  components  adequately 
documented  by  control  drawings 

c.  Proprietary  designs  or  data  not  used  except  where  specifically  authorized 

d.  Design  details  are  completely  disclosed  (DoD-D-1000,  level  3  drawings)  and  are  fully  specified 
with  dimensions  and  tolerances.  Geometric  dimensioning  and  tolerancing  used  to  describe 
replaceable  unit  interfaces. 

e.  Design  appears  to  be  producible  on  a  competitive,  industrial  basis 

f.  Components  and  assemblies  anticipated  to  be  reprocured  as  spare  parts  are  physically  and 
functionally  interchangeable.  Documentation  to  have  classified  characteristics  (DoD-STD-2101) 
and  corresponding  inspection  requirements. 

g.  Printed  circuit  boards  and  flexible  circuitry  designed  to  MIL-STD-275  and  designed  to  be 
producible  to  MIL-P-55110  using  WS-6536 

h.  Documentation  describing  functional  subassemblies  (e.g.  PWB  assemblies)  and  units;  assemblies 
include  performance  requirements 

i.  Documentation  includes  appropriate  quality  assurance  (i.e.  inspection  and  test)  requirements. 

j.  Documentation  includes  appropriate  process  control  (e.g.  WS-6536)  and  workmanship  (e.g. 
MIL-STD-454,  Req  9)  requirements. 

k.  Documentation  meets  military  standards  and  specifications  (i.e.  DoD-D-1000,  DoD-STD-100, 
ANSI-Y14.5,  MIL-S-83490,  MIL-STD-490). 


11.9  INTEGRATED  LOGISTIC  SUPPORT 
11.9.1  Definition 

Integrated  logistic  support  is  a  composite  of  all  the  logistic  support  considerations  necessarv  to 
assure  the  effective  and  economical  support  of  a  svstem/equipment  throughout  its  life  cvcle:  it  is  an 
integral  part  of  the  system  acquisition  process. 


11.9.2  Integrated  Logistic  Support  Program  Policy  (NAVMATINST  4000.20) 

The  requirements  of  NAVMATINST  4000.20,  ILS  Planning  Policy,  are  described  by  paragraph 
3,  following. 


11.9.3  Integrated  Logistic  Support  Program  Elements 


11.9.3.1  Maintenance  Plan.  This  element  includes  the  description  of  the  requirements  and  tasks  to 
be  accomplished  for  achieving,  restoring,  or  maintaining  the  operational  capability  of  a 
system/equipment.  The  basis  for  the  maintenance  plan  is  the  maintenance  concept  which  describes 
the  manner  in  which  a  system/equipment  will  be  maintained  and  supported.  The  maintenance  concept 
can  involve  the  following  maintenance  levels: 

a.  Organizational  maintenance — Planned  or  corrective/ unscheduled  maintenance  by  the  operational 
unit 

b.  Intermediate  maintenance — Submarine/destroyer  tenders  (AS/ AD),  repair  ships  (AR),  shore 
activities,  submarine  support  facilities 

c.  Depot  maintenance — Organic  (Navy/DoD)  or  commercial  designated  overhaul  points  (DOP) 

d.  Direct  Fleet  support — Direct  technical  assistance  to  organization  and  intermediate  levels  (e.g., 
mobile  technical  units). 

11.9.3.2  Manpower  and  Personnel.  This  includes  requirements  for  the  numbers  (officers  and  enlisted 
personnel)  and  skills  (classifications)  of  personnel  to  operate  and  maintain  the  system/equipment. 

11.9.3.3  Supply  Support.  This  addressec  the  requirements,  including  the  initial  operating  requirements, 
for  provisioning  material  needed  at  all  maintenance  levels  including  spare  and  repair  parts  and 
consumables.  Supply  support  considerations  include  expected  frequency  of  repair  and  need  for  early 
supply  support,  phased  provisioning  or  prescreening,  anticipated  contractor  depot  support,  and  repairable 
material  program.  Supply  support  plan  decisions  are  reflected  in  various  items  of  provisioning  technical 
documentation  (e.g. .provisioning  parts  list,  common  and  bulk  items  list,  interim  support  items  list, 
long  lead  time  items  list,  and  tools  and  test  equipment  list). 

11.9.3.4  Support  and  Test  Equipment  (S&TE).  This  includes  requirements  for  special  and  standard 
test  equipment,  special  and  standard  tools,  fixtures,  etc.  and  requirements  for  maintenance  and  calibration 
of  these  devices. 

11.9.3.5  Training  and  Training  Devices.  This  addresses  requirements  for  training  of  personnel, 
including  both  initial  and  follow-on/refresher  training,  as  well  as  requirements  for  training  materials 
(e.g.,  instructor/lesson  training  course  guides,  student’s  training  course  guides)  and  for  training  devices 
or  equipment  including  their  development,  fabrication,  maintenance,  and  other  support. 

11.9.3.6  Technical  Data.  This  includes  requirements  for  technical  manuals  or  other  documents  (e.g., 
maintenance  requirements  cards)  for  the  organizational,  intermediate,  and  depot  maintenance  levels, 
including  the  detailed  format  and  technical  contents;  the  validation  and  verification  and  the  life-cycle 
maintenance;  engineering  drawings  and  product  specifications  acquisition  and  maintenance  for  use  in 
procurement  of  systems/equipment  and  their  spare  parts  and  for  depot  repair  operations. 
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11.9.3.7  Computer  Resources  Support.  This  includes  requirements  for  all  computer  equipment  and 
computer  software  and  the  requirements  for  their  support. 

11.9.3.8  Packaging,  Handling,  Storage,  and  Transportation  (PHST).  This  includes  requirements 
for  preservation,  packaging,  packing,  and  marking,  including  special  containers  to  prevent  damage 
during  shipment  and  storage;  jigs,  fixtures,  or  other  equipment  needed  for  movement  during  shipment; 
space  and  environment  for  storage,  including  storage  related  maintenance;  primary  and  alternate  modes 
of  transportation. 

11.9.3.9  Facilities.  This  addresses  requirements  for  the  construction  or  modification  of  new  or  existing 
facilities  of  all  types,  including  assembly  maintenance,  and  for  storage  facilities  both  afloat  and  ashore. 


11.10  DOD/NAVY  NOSC  PRODUCT  ASSURANCE  POLICY  OVERVIEW 

The  following  provides  an  overview  of  the  more  significant  product  assurance  policy  directives 
affecting  the  elements  discussed  in  the  following  sections.  For  a  more  complete  list,  it  is  recommended 
that  the  reader  refer  to  TD  432. 

11.10.1  Reliability  and  Maintainability  Policy  Directives 

a.  DoD  Directive  5000.50,  Reliability  and  Maintainability 

Establishes  DoD  policies  and  responsibilities  for  the  reliability  and  maintainability  of  defense 
systems,  subsystems,  and  equipment 

b.  NAVMATINST  3000.1,  Reliability  of  Naval  Material 

Establishes  Navy  policy  for  the  acquisition  and  deployment  of  reliable  material.  Key  technical 
requirements  imposed  on  the  design  agent  include  invocation  of  engineering  and  manufacturing 
disciplines  and  controls;  integrated  test  program;  reliability  test  program;  continous  reliability 
assessment;  reporting,  analysis,  and  correction  of  failures;  contractor  control 

c.  NAVMATINST  3000.2,  Operational  Availability  of  Weapon  Systems  and  Equipments; 
definitions  and  policy 

Establishes  operational  availability  (A0)  as  the  primary  measure  of  material  readiness  for  Navy 
weapons  systems  and  equipment 

d.  NAVSEAINST  3900.2,  Reliability  and  Maintainability  Program  of  the  Naval  Sea  Systems 
Command  for  Design,  Development  and  Acquisition  (Non-Nuclear) 

Implements  NAVMATINST  3000.1  policy,  with  further  expansion  of  certain  requirements, 
for  NAVSEA  programs.  Includes  maintainability  requirements  in  consonance  with  those  for 
reliability 

e.  NAVSEA  OD  46574B  Weapons  and  Combat  Systems  Quality  Assurance  Requirements  for 
Shore  Stations  and  Engineering  Agents 

Establishes  overall  product  assurance  program  requirements,  including  reliability  and 
maintainability,  for  NAVSEA’s  weapons  and  combat  systems  development,  production,  and 
in-service  support  activities 


f.  NAVELEXINST  4858.2,  Naval  Electronics  Systems  Command  Reliability  Program 

Implements  NAVMATINST  3000.1  and  prescribes  basic  policy  for  development  and 
implementing  NAVSPAWAR’s  reliability  program  requirements  and  provides  guidance  and 
specific  direction 

g.  NAVELEXINST  4858.3,  Naval  Electronic  Systems  Command  Maintainability  Program 

Delineates  the  basic  policies,  specific  directions,  and  designation  of  responsibilities  in  developing 
and  implementing  NAVSPAWAR’s  maintainability  program  requirements 

h.  NAVAIRINST  13070.2,  Policy  for  Reliability  and  Maintainability  (R&M)  of  Naval  Aeronautical 
Systems  and  Equipment 

Implements  NAVMATINST  3000.1  and  promulgates  NAVAIR’s  policy  governing  reliability 
and  maintainability  (R&M)  programs  and  delineates  responsibilities 

i.  NOSCINST  4855.1,  NOSC  Product  Assurance  Program 

Requires  department  heads  to  ensure  that  the  following  actions  are  accomplished: 

NOSC  TD  432,  Product  Assurance  Requirements  Guide  for  Naval  Ocean  Systems  Center 
Projects,  used  when  planning  requirements,  including  reliability  and  maintainability 

Adherence  to  NAVMAT/SYSCOM  policy  requirements  to  be  verified  by  periodic  internal 
program  reviews 

Formal  design  reviews  (PDR,  CDR,  DCR)  conducted 

Records  of  internal  program  reviews  and  design  reviews  maintained 

Product  assurance  program  critique  during  DRC  review. 

j.  NOSC  TD  870,  Product  Assurance  Program  Requirements  Manual  for  Naval  Ocean  Systems 
Center  Projects 

Establishes  product  assurance  program  content  requirements  for  hardware  (including  reliability 
and  maintainability)  and  computer  software  aspects  of  NOSC  projects  in  all  phases  of  engineering 
development,  production,  or  in-service  operation;  applies  to  all  NOSC  Fleet  systems/equipment 
projects;  implements  NAVSEA  OD46574B. 

11.10.2  System  Safety  Policy  Directives 

a.  DoD  Directive  5000.36,  System  Safety  Engineering  and  Management 

Establishes  policy  for  the  Department  of  Defense  system  safely  engineering  and  management 
programs 

b.  NAVMATINST  5100.6,  System  Safety  Program;  implementation  of 

Provides  Navy  policy  and  guidance  for  the  incorporation  of  ~afety  programs  in  the  process 
of  acquiring  systems  and  equipment.  Requires  the  SYSCOMs  to  ensure  that  system  safety 
programs,  based  on  MIL-STD-882  (System  Safety  Program  Requirements)  are  incorporated 
into  all  system  acquisitions 

c.  NAVSEAINST  5100.6,  Safety  Program;  command  policy  and  responsibilities  concerning 

Defines  NAVSEA  policy  for  the  conduct  of  system  safety  and  establishes  program 
responsibilities  and  provisions;  Weapons  Systems  Explosive  Safety  Resiew  Board;  hazards 
of  electromagnetic  radiation  to  ordnance  responsibility 

d.  NAVSEAINST  5100.12,  System  Safety  Program  for  Ships,  Ship-borne  Systems  ard  Subsystems 
and  Equipment 
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Establishes  and  promulgates  NAVSEA  policy  for  the  implementation  of  M1L-STD-882  for 
ship  system  safety  programs 

e.  NAVALEXINST  5100.5,  System  Safety  Program;  implementation  of 

Establishes  NAVSPAWAR’s  policy  for  implementation  of  NAVMATINST  5100.6  and  MIL- 
STD-882.  Defines  responsibilities  for  system  safety  program  approval  and  monitoring 

f.  NAVAIRINST  5100.3,  System  Safety  Policies,  Objectives  and  Responsibilities 

Implements  NAVMATINST  5100.6  and  establishes  policies,  objectives,  and  responsibilities 
for  incorporating  system  safety  into  NAVAIR  systems/equipment  acquisitions 

g.  NOSCINST  4855.1,  NOSC  Product  Assurance  Program 

Requires  that  NOSC  TD  432  be  used  when  planning  product  assurance  program  requirements, 
including  system  safety 

h.  NOSC  TD  870,  Product  Assurance  Program  Requirements  Manual  for  Naval  Ocean  Systems 
Center  Projects 

Establishes  product  assurance  program  content  requirements,  including  system  safety,  for  NOSC 
projects 

i.  NOSCINST  5100.3,  System  Safety  Program;  implementation  of 

States  NOSC’s  policy  for  implementation  of  system  safety  into  the  development  and  acquisition 
of  Navy  systems  and  equipment. 

11.10.3  Human  Factors  Policy  Directives 

a.  NAVMATINST  3900.3,  Human  Factors 

Establishes  Navy  policy  and  requirements  to  ensure  adequate  development  of  the  human  factors 
aspects  of  systems/equipment.  Invokes  the  requirements  of  MIL-H-46855 

b.  NOSCINST  4855.1,  NOSC  Product  Assurance  Program 

Requires  that  NOSC  TD  432  be  used  when  planning  product  assurance  program  requirements, 
including  human  factors 

c.  NOSC  TD  870,  Product  Assurance  Program  Requirements  Manual  for  Naval  Ocean  Systems 
Center  Projects 

Establishes  product  assurance  program  content  requirements,  including  human  factors,  for 
NOSC  projects. 

11.10.4  Quality  Assurance  Policy  Directives 

a.  DoD  Directive  4155.1,  Quality  Program 

Establishes  DoD  quality  program  policies  for  products  and  services  and  assigns  responsibilities 

b.  SECNAV  Instruction  4855.1,  Quality  Assurance  Program 

Implements  DoD  Directive  4155.1,  assigns  basic  responsibilities  for  administration  of  quality 
assurance,  and  designates  a  central  management  focal  point  within  the  Department  of  the  Navy 


c.  NAVMATINST  4855.1,  Quality  Assurance  Policy  for  the  Naval  Material  Command 

Establishes  Navy  policies,  principles,  and  responsibilities  for  implementation  and  monitoring 
weapon/support  systems  and  equipment  quality  assurance  programs 

d.  NAVELEXINST  4855.2,  Naval  Electronic  Systems  Command  Quality  Assurance  Program 

Implements  NAVMATINST  4855.1  and  NAVSPAWAR’s  systems  effectiveness  program 
policies  in  the  quality  assurance  area.  Key  elements  of  the  program  include  effective  quality 
assurance  actions  applied;  actions  coordinated  under  quality  assurance  program;  planning  and 
control  exercised 

e.  NAVSEAINST  4855.5,  Quality  Assurance  Program  of  the  Naval  Sea  Systems  Command 

Implements  DoD  and  Navy  policies  regarding  quality  assurance.  Key  elements  of  NAVSEA 
policy  include  quality  assurance  program  planning  throughout  life  cycle;  continuous  program 
assessment;  programs  required  for  engineering  agent  activities;  quality  requirements  for  IFBs, 
RFPs,  contracts;  contractors  responsible  for  quality  of  products;  program  for  ensuring  quality 
of  technical  data 

f.  NAVAIRINST  5400.23,  Quality  Assurance  Program  of  the  Naval  Air  Systems  Command 

Implements  DoD,  SECNAV,  and  NAVMAT  policies  and  establishes  NAVAIR  program 
responsibilities  and  requirements,  including: 

Quality  assurance  requirements  for  all  material,  data,  services— conformance  to  requirements 
to  be  ensured 

User  dissatisfaction,  mission  ineffectiveness  to  be  prevented 

g.  NOSCINST  4734.1,  Repair/Calibration  of  Center  Test,  Measuring,  and  Diagnostic  Equipment 

Requires  all  meteorology  and  test,  measuring  and  diagnostic  equipment  used  for  quantitative 
measurements  to  be  calibrated  in  accordance  with  the  stated  policy 

h.  NOSCINST  4855.1,  NOSC  Product  Assurance  Program 

Requires  that  NOSC  TD  432  be  used  when  planning  product  assurance  program  requirements, 
including  quality  assurance 

i.  NOSC  TD  870,  Product  Assurance  Program  Requirements  Manual  for  Naval  Ocean  Systems 
Center  Projects 

Establishes  product  assurance  program  content  requirements,  including  quality  assurance,  for 
NOSC  projects. 

11.10.5  Configuration  Management  Policy  Directives 

a.  DoD  Directive  5010.19,  Configuration  Management 

Establishes  Department  of  Defense  policies  governing  the  configuration  management  of 
systems/equipment  and  other  designated  material  items 

b.  NAVMATINST  4130.1,  Joint  DoD  Services/ Agency  Regulation — Configuration  Management 

A  joint  regulation  endorsed  by  all  DoD  components  (Army,  Navy,  Air  Force),  the  Defense 
Supply  Agency,  the  National  Security  Agency,  the  Defense  Communications  Agency,  and  the 
Defense  Nuclear  Agency.  Establishes  following  requirements: 

Configuration  management  programs  required 


Configuration  identification 
Configuration  control 
Configuration  status  accounting 
Configuration  audits  (FCA,  PCA) 

Functional,  allocated,  product  baselines  required 
Engineering  change  proposals,  deviations,  wrivers  described 
Configuration  control  board  operation  described 

c.  NAVSEAINST  4130. 10,  Configuration  Control  Board  Operations  for  Systems  and  Equipments; 
establishment  of 

Establishes  configuration  control  board  (CCB)  operation  and  procedures  for  the  review  and 
processing  of  engineering  changes 

d.  NOSCINST  4855.1,  NOSC  Product  Assurance  Program 

Requires  that  NOSC  TD  432  be  used  when  planning  product  assurance  program  requirements, 
including  configuration  management 

e.  NOSC  TD  870,  Product  Assurance  Program  Requirements  Manual  for  Naval  Ocean  Systems 
Center  Projects 

Establishes  product  assurance  program  content  requirements,  including  configuration 
management,  for  NOSC  projects. 


11.10.6  Design  Review  Policy  Directives 

a.  NAVMATINST  3000.1,  Reliability  of  Naval  Material 

Establishes  Navy  policy  for  the  acquisiton  and  deployment  of  reliable  material.  Includes  require¬ 
ments  for  design  reviews  to  be  held  in  connection  with  contractor  development  programs,  e.g. 

Preliminary  design  review 
Critical  design  review 
Design  certification  review 

b.  NAVSEAINST  9070.5A,  Design  Reviews  of  Naval  Sea  Systems  Command  Acquisition 
Programs;  Policy  and  Procedures  for 

Establishes  requirement  that  all  systems/equipment  acquisition  programs  to  include  design 
reviews,  as  follows: 

Major  system/equipment  acquisitions 
System  design  review 
Preliminary  design  review 
Critical  design  review 
Additional  reviews  which  may  be  required 
Special  purpose  design  review 
Preproduction  design  review 

c.  NOSCINST  3912. IB,  Design  Review  Committee;  Establishment  of 

Establishes  the  NOSC  design  review  committee  (DRC)  which  provides  advice  to  the  technical 
director  regarding  the  development  and  readiness  for  production  of  NOSC-developed 
systems/equipment.  Requires  the  following  reviews  during  the  full-scale  development  phase: 

Preliminary  design  review 
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Critical  design  review 
Design  certification  review 

d.  NOSCINST  4855-1,  NOSC  Product  Assurance  Program 

Requires  that  preliminary,  critical,  and  design  certification  reviews  be  conducted 

e.  NOSC  TD  870,  Product  Assurance  Program  Requirements  Manual  for  Naval  Ocean  Systems 
Center  Projects 

Requires  that  formal  design  reviews  be  conducted,  to  include: 

System  requirements  review  (conceptual  phase) 

System  design  review  (validation  phase) 

Preliminary  design  review  (full-scale  development  phase) 

Critical  design  review  (full-scale  development  phase) 

Design  certification  review  (full-scale  development  phase). 

11.10.7  Design  Documentation  Policy  Directives 

NOSC  TD  870,  Product  Assurance  Program  Requirements  Manual  for  Naval  Ocean  Systems  Center 
Projects 

Establishes  product  assurance  program  content  requirements,  including  design  documentation 
provisions,  NOSC  projects.  Provides  application  guidance  for  DoD-D-1000  levels  1,  2,  and  3 
drawings  and  establishes  requirements  for  each  level.  Establishes  requirements  for  specifications. 
Establishes  requirements  for  classification  of  characteristics. 

11.10.8  Integrated  Logistic  Support  Policy  Directives 

a.  DoD  Directive  5000.39,  Acquisition  and  Mangement  of  Integrated  Logistic  Support  for  Systems 
and  Equipment 

Establishes  DoD  policy  and  assigns  responsibility  for  integrated  logistic  support  (ILS)  including 
the  acquisition  of  ILS  as  an  integral  part  of  the  systems  and  equipment  acquisition  process 

b.  SECNAVINST  5000.39,  Acquisition  and  Management  of  Integrated  Logistic  Support  (ILS) 
for  Systems  and  Equipment 

Implements  DoD  Directive  5000.39  and  establishes  Navy  policy  and  assigns  responsibility  for 
ILS  within  the  Navy 

c.  NAVMATINST  4000.20,  ILS  Planning  Policy 

Establishes  Navy  policies  and  principles  for  the  life-cycle  support  of  systems/equipment. 
Identifies  the  principal  elements  of  ILS  including: 

Maintenance  planning 
Support  and  test  equipment 
Supply  support 
Transportation  and  handling 
Technical  data 
Facilities 

Personnel  and  training 

Logistic  support  resource  funds 

Logistic  support  management  information 


d.  NAVSEAINST  4105.1,  Integrated  Logistic  Support  (ILS);  policy,  responsibilities  and  planning 

Establishes  NAVSEA  ILS  policy  in  accordance  with  NAVMATINST  4000.20,  assigns 
responsibility  for  ILS  actions  and  products,  and  establishes  standard,  detailed  requirements 
for  preparation  and  modification  of  ILS  planning  documents.  Identifies  the  particular  ILS 
program  requirements  for  each  of  the  life-cycle  phases 

e.  NAVAIRINST  4000.2,  ILS  Planning  Procedures 

Establishes  policy  and  responsibilities  for  the  application  of  ILS  program  requirements  and 
related  management  procedures  within  NAVAIR.  Implements  NAVMATINST  4000.20  as  the 
guiding  document  for  NAVAIR  ILS  planning  and  implementation 

f.  NAVELEXINST  4000.6,  Integrated  Logistic  Support  (ILS);  policy  and  responsibilities 

Establishes  NAVSPAWAR’s  ILS  policy  in  accordance  with  NAVMATINST  4000.20,  assigns 
responsibilities  for  ILS  actions  and  products,  and  establishes  standard  ILS  certification 
requirements  for  material 

g.  NOSCINST  4855.1,  NOSC  Product  Assurance  Program 

Requires  that  NOSC  TD  432  be  used  when  planning  product  assurance  program  requirements, 
including  ILS 

h.  NOSC  TD  870,  Product  Assurance  Program  Requirements  Manual  for  Naval  Ocean  Systems 
Center  Projects 

Establishes  product  assurance  program  content  requirements,  including  ILS,  for  NOSC  projects. 
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SECTION  12 

TEST  AND  EVALUATION 
F.  Held,  Code  954 


12.1  INTRODUCTION 


12.1.1  References 


System  Acquisition  T&E  Handbook,  NAVSEA 
T&E  Handbook,  3rd  edition,  NAVSPAWAR 


12.1.2  Outline 
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12.1.3  Summary 


Successful  test  and  evaluation  (T&E)  is  the  major  determining  factor  in  program  acquisition 
milestone  decisions.  This  is  also  probably  true  at  any  time,  but  certainly  during  the  acquisition  process. 
It  is  not  the  intention  of  this  presentation  to  make  test  directors  of  each  of  you,  but  rather  to  present 
and  review  pertinent  T&E  information  for  you  as  program  managers  so  you  can  conduct  successful 
programs.  Required  T&E  documents  are  identified,  including  test  and  evaluation  master  plans  (TEMPs), 
test  plans,  procedures,  and  reports.  Considerations  to  be  evaluated  during  site  selections  are  listed. 
The  purposes  for  a  technical  evaluation  (TECHEVAL)  and  operational  evaluation  (OPEVAL)  are 
presented,  as  well  as  the  OPEVAL  readiness  criteria.  T&E  services  and  capabilities  provided  by  Support 
Engineering,  Code  95,  are  reviewed  and  examples  of  some  NOSC  programs  are  discussed. 


12.2  DEFINITIONS 


ACAT 

Acquisition  category 

D/V  (D&V) 

Demonstration  and  validation  phase  I 

DT  I 

Developmental  testing  during  D/V  phase 

DT  II 

Developmental  testing  during  full-scale  development,  phase  II 

ADM 

Advanced  development  model 

EDM 

Engineering  development  model 

FSD 

Full-scale  development 

FSP 

Full-scale  prototype 

FOT&E 

Follow-on  T&E 

MS 

Milestone 

OT 

Operational  test 

OPEVAL 

Operational  evaluation 

TECHEVAL 

Technical  evaluation 

PAT&E 

Production  acceptance  T&E 

12.3  T&E  DURING  THE  ACQUISITION  PROCESS 

There  are  four  phases  (O,  I,  II,  and  III)  during  the  acquisition  process  which  you  have  probably 
heard  many  times  by  now.  The  decision  to  proceed  from  one  phase  to  the  next  is  considered  a  milestone 
(MS),  and  are  numbered  by  the  phase  you  are  deciding  to  enter.  The  results  of  T&E  are  the  major 
factors  in  determining  program  milestone  decisions. 

12.3.1  Milestone  I 

There  is  little  if  any  T&E  to  support  MS-I,  usually  studies,  unless  the  project  is  using  some  new 
state-of-the-art  technology.  If  so,  some  tests  may  be  required. 
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12.3.2  Milestone  11 


MS-II  is  based  upon  the  DT-1  and  OT-I  performed  during  the  D/V,  phase  I.  Again,  the  extent 
of  testing  is  determined  by  the  technology  used.  Current  technology  requires  less  T&E  than  “state-of- 
the-art.”  These  tests  are  usually  conducted  on  advanced  development  models  (ADM)  and  require 
appropriate  test  documentation. 


12.3.3  Milestone  III 

The  decision  to  go  into  production,  MS-III,  is  based  on  the  results  of  DT-II  and  OT-1I  testing. 
These  tests  are  performed  on  full-scale  prototype  units,  or  engineering  development  models  (EDMs). 
These  units  should  be  similar  to  the  production  units  to  reduce  the  risks  of  having  the  production  units 
fail  and  eliminate  some  of  the  first  article  tests.  You  will  notice  I  have  mentioned  units — multiple  test 
units,  or  parts  of  a  unit,  are  required  as  several  tests  are  conducted  in  parallel  (i.e.,  environmental, 
performance,  reliability,  etc.),  and  some  tests  are  destructive  (shock,  salt  spray,  fungus,  etc.).  The  size 
of  the  system  (program  product),  costs,  and  schedule  will  all  have  a  bearing  upon  the  quantity  of  test 
units  planned. 

12.3.3.1  TECHEVAL.  The  last  test  of  DT-II  is  usually  a  TECHEVAL  for  which  you  must  develop 
specific  documentation.  The  purpose  of  a  TECHEVAL  is  to  accomplish  the  following: 

Verify  that  the  system  meets  the  technical  performance  requirements 

Verify  compatibility  with  other  systems  and  environment 

Verify  that  the  system  is  operationally  satisfactory  and  ready  for  OPEVAL. 

12.3.3.2  OPEVAL.  OT-II  consists  of  an  operational  test  conducted  by  an  independent  agency, 
OPTEVFOR,  using  military  operators  of  quantity  and  rate,  as  those  planned  for  the  Fleet.  This  Navy 
crew  will  have  been  trained  by  the  curriculum  developed  for  this  program.  The  OPEVAL  readiness 
criteria  include  the  following: 

TEMP  current  and  approved 

DT-II  completed  and  reports  published 

All  DT&E  objectives  and  performance  thresholds  met 

Engineering  complete  and  “ilities”  satisfactory 

System  expected  to  perform  successfully  in  OPEVAL 

System  maintenance  documents,  logistics  support  plan,  failure  mode  and  effects  analysis  (FMEA), 
life-cycle  cost  (LCC),  and  logistic  support  analysis  supplied  to  OPTEVFOR 

Space  parts  available  for  OPTEVFOR 

Navy  training  plan  approved  and  provided  OPTEVFOR 

OPEVAL  crew  consists  of  numbers  and  rates  planned  for  Fleet,  and  have  completed  training 
All  OPEVAL  resources  listed  in  the  TEMP  available 
The  safety  program  satisfactorily  completed. 


12.4  CRITICAL  T&E  DOCUMENTS 


The  test  and  evaluation  master  plan  (TEMP)  is  probably  the  most  important  program  T&E 
document,  followed  by  the  performance  specification.  Specifications  may  be  considered  T&E  documents 
as  they  contain  test  requirements.  Most  of  the  test  requirements  are  identified  in  the  TEMP  and/ or 
system  specification.  A  T&E  management  plan  may  be  required,  depending  upon  the  size  of  the  program. 
For  smaller  projects,  information  (T&E  responsibilities)  is  contained  in  the  Navy  test  plan  along  with 
integration  and  performance  test  plans. 

Environmental  test  plans  and  procedures  are  required  to  verify  the  requirements  identified  in  the 
quality  section  of  the  specifications.  A  test  requirement  document  may  be  required,  again,  depending 
upon  the  size  of  the  program  and  the  number  of  official  specifications.  For  the  larger  programs,  a 
tests  requirements  document  (TRD)  is  required,  especially  if  there  are  a  number  of  applicable 
specifications.  This  document  lists  all  the  test  requirements  and  references  the  specification  or 
specifications  from  which  the  requirement  came.  A  TRD  makes  it  much  easier  to  get  a  handle  on  the 
test  requirements  and  to  be  sure  that  none  of  them  fall  through  the  cracks. 

Another  critical  T&E  document  is  the  TECHEVAL  plan.  It  along  with  procedures  is  necessary 
for  a  successful  TECHEVAL.  Many  of  the  other  previously  identified  plans  will  also  require  procedures. 
For  smaller  projects,  the  plan/procedures  can  be  one  document.  Of  course  all  of  these  tests  require 
reports  documenting  the  results.  Test  reports  provide  the  supporting  data  to  keep  a  project  moving 
from  phase  to  phase  at  the  various  milestones. 


12.5  TEST  PLANNING 

12.5.1  Test  and  Evaluation  Master  Plan 

One  of  the  first  test  plans  to  be  developed  is  the  TEMP.  This  should  be  completed  by  MS-I,  or 
early  in  the  D/V  phase.  The  purposes  of  a  TEMP  include  the  following: 

Define  and  control  adequate  T&F. 

Identify  and  reserve  special  resources 

Document  major  agreements  between  the  SYSCOM  and  OPTEVFOR. 

The  TEMP  consists  of  the  following  items: 

Cover  Page 
TEMP  number 
Title 

ACAT  number 
Submittal  date 

Review  and  approval  signature  lines 
Part  I 

Major  points  of  contact 
System  description 
What  it  does  for  the  Fleet 
Financial  summary 

T&E  critical  issues,  cross-referenced  to  tests  in  parts  II  and  IV 

Specific  performance  requirements  for  OT&E  and  DT&E,  acceptable  criteria  and  failure 
definitions. 
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Part  II 

Program  structure,  showing  DT  and  OT  periods,  milestones,  test  articles,  and  deliveries 
Part  III 

DT&E  outline,  describing  each  major  DT  period,  DT&E  to  date,  and  future  DT&E  related  to 
critical  issues 

Part  IV 

OT&E  outline,  describing  OT,  and  identification  of  DT&E  results  to  be  used  for  OT&E 
Part  V 

Production  acceptance  and  evaluation 
Pan  VI 

T&E  resource  summary. 

12.5.2  Test  Plans 

As  usual  there  are  a  number  of  formats  for  test  plans,  or  “many  ways  to  skin  a  cat.”  This  outline 
includes  all  the  required  elements  and  will  give  you  a  check  list  for  any  plan: 

Introduction-objectives 

Scope 

Test  support 

Installation  and  checkout  (I&C) 

Test  conditions  and  criteria 
Test  methods 
Test  schedule 

Test  documentation,  procedures,  and  reports. 

12.5.3  Site  Selection 

The  selection  of  a  test  site  is  dependent  upon  the  type  of  tests  required  to  be  performed.  It  is  a 
process  of  getting  the  best  combination  of  environments  while  considering  costs. 

Why  use  a  land-based  site  versus  at  sea  testing: 

Less  expensive  and  enables  controlled  tests 
Reduces  risks  for  final  at-sea  tests 
Frees  up  ships 

Wide  range  of  permutations  available 

Knowledgable  test  team  available  and  provides  training  for  ship  crews 

Allows  stressing  the  system 

OPTEVFOR  involvement  (upon  PA  request) 

Validation  of  I&C  procedures. 


12.6  CONDUCTING  THE  TESTS 

The  testing  periods  are  usually  just  prior  to  the  milestone  decisions.  There  is  very  little,  if  any. 
testing  before  MS-I.  This  is  usually  a  study  and  analysis  phase.  If  there  is  a  new  technology  or  questionable 
state-of-the-art  process  being  used,  some  testing  may  be  required  at  this  time.  Again  the  amount  of 


testing  on  advanced  development  models  (ADMs)  is  dependent  on  the  type  of  technology  being  used — 
little  testing  for  current  technology  and  more  for  newer  technologies.  The  full-scale  development  phase 
is  the  primary  test  period.  This  is  where  the  developing  agency  must  determine  if  the  system  is  ready 
for  OPEVAL  and  satisfies  the  criteria  of  ready  for  OPEVAL. 

12.6.1  Test  Team  Section 

When  selecting  a  test  team,  maintain  a  continuity  of  personnel.  That  is,  use  some  of  the  people 
that  developed  the  test  requirements  document  and  the  test  plans.  Also,  get  support  from  the  developer. 
Test  team  members  should  have  some  previous  experience,  at  least  the  key  personnel.  Include  some 
equipment  specialists,  and  invite  OPTEVFOR  to  participate  or  observe. 

12.6.2  Conduct 

When  conducting  the  tests  know  what  data  is  required  to  verify  performance.  Be  familiar  with 
the  procedures;  they  should  be  dry  run  prior  to  the  official  test  run.  Provide  data  sheets,  and  use  them! 
Maintain  an  operating  log  of  all  actions. 

12.6.3  Reporting 

A  trick  of  making  sure  you  get  the  necessary  data  is  to  start  the  test  report  before  you  start  the 
test.  Once  you  see  the  holes  in  the  report  you  can  determine  the  data  required  to  fill  them.  Collect 
all  the  data  during  and  after  testing,  and  provide  copies  of  supporting  data  with  the  test  report.  Explain 
how  the  data  analysis  was  accomplished,  and  make  recommendations. 

12.6.4  Environmental  Testing 

This  consists  of  testing  the  system  under  the  various  conditions  it  would  encounter  in  actual 
operation.  Environmental  tests  cover  the  following. 

Low  temperature 
Humidity 
Sunshine 
Wind  velocity 
Hydrostatic  pressure 
Dust 

Thermal  shock 
Shock 

12.6.5  NOSC  Capabilities 

The  Electronic  and  Environmental  Test  Branch,  Code  951,  has  the  capability  to  conduct  first  article 
acceptance  and  performance  testing  for  almost  any  radio  receiver  or  transmitter,  and  for  much  other 
electronic  equipment. 

Four  shielded  enclosures  are  available  to  enable  testing  without  external  interference.  Two  enclosures 
are  ducted  together  to  enable  the  test  stimulus  or  equipment  to  be  shielded  and  shielded  from  the  test 
unit.  The  largest  enclosure  is  20  by  24  by  15  feet. 


High  temperature 

Salt  fog 

Fungus 

Icing 

Altitude 

Rain 

Transportability 

Vibration 
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Three  vibration  machines  for  electronic  equipment  testing  are  available  with  loads  up  to  620  pounds  < 

and  frequencies  from  5  to  2,500  Hz. 

Two  mechanical  vibration  machines  are  available,  primarily  for  mechanical  shipboard  vibration  j 

testing.  They  have  load  capacity  up  to  10,000  pounds,  and  frequencies  from  5  to  40  Hz.  J 

Shock  machines  are  capable  of  loads  up  to  5,600  pounds  and  up  to  2,000  g’s.  | 

Five  climatic  chambers  are  available  to  simulate  environments  of  temperature,  humidity,  altitude,  ! 

and  sea  pressure.  ! 

The  Systems  Test  and  Evaluation  Branch,  Code  954,  provides  the  following  services:  j 

Develop  quality  sections  of  specifications  J 

Draft  TEMPs  J 

Develop  test  plans  and  procedures  j 

Conduct  tests 

Develop  test  reports  ! 

Test  consultation 

Review  of  procurement  packages  (T&E  aspect).  ! 


12.7  PROJECT  EXAMPLES 

12.7.1  SCIACT  (Secure  Communication  Integration  of  the  AN /G  SC -40  Command 
Post  Terminal) 

The  SCIACT  program  developed  a  T&E  management  plan.  This  identified  the  tests,  test  schedule, 
test  organization  and  responsibilities,  test  personnel,  and  test  facilities.  It  listed  these  tests: 

Environmental 

EMI 

EMP 

Tempest 

System  integration 
System  acceptance. 

A  specific  test  plan  and  procedure  were  written  for  each  test.  The  test  personnel  participated  in  the 
development  of  these  documents  and  were  thoroughly  familiar  with  them  when  conducting  the  tests. 
The  program  was  completed  on  schedule  and  performed  satisfactorily  in  the  field.  Its  success  is  attributed 
to  good  planning,  competent  and  trained  personnel,  good  test  coordinator,  and  sufficient  funding  to 
do  the  task. 


12.7.2  TRIDENT  Integrated  Radio  Room 

The  integrated  radio  room  (IR R)  is  another  project  that  developed  a  good  T&E  program.  This 
was  a  dual  development  FSP  program  from  which  a  production  contractor  was  to  be  selected  after 
Navy  independent  testing.  Due  to  overruns  and  schedule  slippage  the  dual  independent  Navy  tests  were 
not  conducted.  The  Navy  tested  each  system  (neither  was  completed)  simultaneously  at  each  contractor’s 
facility  with  the  help  of  the  particular  developing  contractors.  The  results  of  their  tests,  plus  their  proposals 
to  complete  the  system  were  evaluated  and  one  contractor  was  selected  for  a  limited  production. 


12-9 


The  first  delivered  unit  underwent  a  complete  independent  Navy  acceptance  and  TECHEVAL. 
These  tests  caused  numerous  hardware  and  software  changes  to  be  made  to  the  system,  and  the  schedule 
slipped  more  than  a  year.  It  did  result  in  an  operational  system  delivered  to  the  first  TRIDENT  submarine. 
Also  much  of  the  schedule  slip  was  due  to  the  hull  delivery. 

12.7.3  Message  Processing  and  Distribution  System  (MPDS) 

This  project  rushed  the  hardware— the  T&E  program,  primarily  test  procedures.  The  terminals 
were  advanced  breadboards  that  were  rushed  into  production.  Likewise,  the  data  acquisition  and 
distribution  unit  laboratory  model  was  integrated  into  a  deliverable  system.  The  reliability  and  MTBF 
was  very  poor — of  course  we  are  talking  about  the  late  1960s  and  early  1970s  when  T&E  was  primarily 
a  production  acceptance  test.  The  poor  reliability  created  a  lot  of  user  dissatisfaction. 
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13.1.3  Summary 


Sec  below. 


13.2  REQUIREMENTS 

To  understand  the  objectives  of  logistics  support  properly,  it  is  necessary  to  examine  briefly  the 
prime  requirements  for  systems  acquisitions.  These  may  be  expressed  simply  as: 

a.  IT  MUST  DO  WHAT  IT  IS  SUPPOSED  TO  DO  WHEN  IT  IS  SUPPOSED  TO  DO  IT.  This 
factor  is  obviously  related  to  system  performance  and  reliability. 

b.  IT  MUST  BE  ADEQUATELY  DOCUMENTED.  Adequately  documented  is  a  term  that  needs 
some  degree  of  logical  interpretation.  Adequately  is  the  key  word  here.  A  full  level  3  documen¬ 
tation  package  may  not  be  required  on  a  “one  of  a  kind”  system  not  intended  for  production. 
However,  a  level  of  documentation  suitable  for  repair,  adjustment,  operation,  special 
transportation  requirements,  safety,  purchasing  repair  parts,  etc.  would  certainly  be  prudent 
and  well  advised. 

c.  IT  MUST  BE  AFFORDABLE.  Logistically  this  term  is  normally  defined  in  terms  of  money 
and  resources  over  the  life  of  the  system. 

d.  IT  MUST  BE  SUPPORTABLE.  The  term  supportable  is  normally  interpreted  as  logistics 
functions.  Unfortunately,  it  is  not  quite  so  obvious  that  logistic  requirements  and  data  cover 
the  three  other  areas  as  well. 

The  logistics  requirements  include  the  need  for  spare  parts.  The  parts  are  identified  in  the 
configuration  management  documentation.  The  number  of  parts  required  is  dependent  upon 
reliability  and  other  logistics  factors.  The  cost  of  the  total  number  of  spare  parts  and  the  other 
logistical  factors  have  major  impacts  on  a  system’s  affordability. 


13.3  IMPORTANT  FACTS 

a.  Out  of  every  $1  that  will  be  spent  on  a  system  acquisition  over  its  LIFE  CYCLE,  75  cents  will 
be  “locked  in”  during  the  demonstration  and  validation  phase. 

b.  Out  of  every  $1  that  will  be  spent  on  a  system  acquisition  over  its  LIFE  CYCLE,  85  cents  will 
be  “locked  in”  during  the  full-scale  development  phase. 

c.  Approximately  75  percent  of  all  Navy  programs  undergoing  a  logistic  review  group  (LRG)  audit 
for  logistics  certification  to  proceed  to  DSARC  fail. 

d.  Approximately  35  percent  fail  on  their  SECOND  attempt. 
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13.4  WHAT  IS  THE  TECHNICAL  LIFE  CYCLE? 


LIFE  CYCLE  OF  TECHNICAL  ACTIVITIES 


System  Phase 

Time  in  Years 

Concept  exploration 

0-2 

Demonstration/ validation 

2-3 

Full-scale  development 

3-6 

Technical  directions  agent  (TDA)  function 

5-11 

5-11 

Production  and  deployment 

3-5 

Operation  and  support 

15-40 

In-service  engineering  agent  (ISEA)  function 

18-45 

18-45 

Total  system  life  cycle  in  years 

23-56 

13.5  CHANGING  SCOPE  OF  LOGISTIC  REQUIREMENTS  AND  PERCEPTIONS 

As  pointed  out  in  the  previous  paragraph,  the  typical  system  life  cycle  will  range  between  a  quarter 
to  a  half  century!  Reference  to  Figure  13.1,  change  in  logistic  requirements  and  perceptions,  graphically 
shows  what  an  impact  the  new  requirements  impose.  Prior  to  1983  the  application  of  the  logistic  support 
analysis  (LSA)  with  its  companion  logistic  support  analysis  record  (LSAR)  was  to  a  large  extent  optional, 
in  the  early  phases,  and  totally  disappeared  shortly  after  deployment.  The  current  directives  introduce 
logistical  considerations  during  the  preconcept  phase  and  maintain  them  into  the  disposal  phase. 

Figure  13.1  shows  a  change  in  the  lines  for  new  scope  during  the  production/deployment  phase. 
This  change  to  an  open  line  represents  a  reduction  of  effort  in  the  LSA  and  LSAR  activity.  A  key 
point  to  remember  is  that  the  logistical  requirement  does  not  go  away  over  the  entire  life  cycle  of  the 
system.  At  the  end  of  the  TDA  function  it  is  transitioned  to  the  ISEA  and  Program  Office  with  the 
LSAR  data  base. 

It  has  frequently  been  said  that  logistical  planning  is  the  function  of  the  prime  hardware  contractor. 
Nothing  could  be  further  from  the  truth.  The  logistical  planning  of  an  acquisition  is  a  government 
responsibility.  True  it  is  accomplished  with  inputs  from  many  sources  including  the  prime  hardware 
contractor,  support  from  service  contractors,  other  government  agencies,  etc.,  but  the  prime  responsibility 
is  that  of  the  acquisition  program.  This  statement  normally  translates  to  a  TDA  function. 


13.6  BASIC  GUIDANCE 

There  are  approximately  340  +  documents  within  the  Navy  that  are  directly  involved  with  the  subject 
of  logistics.  Fortunately,  it  is  highly  unlikely  that  the  majority  of  them  would  be  applied  to  a  single 
acquisition. 
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DISPOSAL 


The  easiest  approach  to  defining  logistical  requirements  of  an  acquisition  is  covered  in  two  basic 
documents. 

a.  MIL-STD-1388-1A,  Logistic  Support  Analysis  (LSA) 

b.  MIL-STD-1388-2A,  DoD  Requirements  for  a  Logistic  Support  Analysis  Record  (LSAR) 

These  two  documents  implement  the  guidelines  and  requirements  established  by  DoD  Instruction 
5000.2,  Major  Systems  Acquisition  Procedures,  and  DoD  Directive  5000.39,  Acquisition  and 
Management  of  Integrated  Logistic  Support  for  Systems  and  Equipment. 

There  will  be  a  section  in  the  course  which  will  give  specific  guidance. 


13.7  WHAT  ARE  THE  LOGISTICS  ELEMENTS? 

There  are  many  different  definitions  as  to  the  organization  of  the  specific  elements  of  logistics. 
The  list  provided  below  is  one  that  all  of  the  services  generally  agree  with.  When  logisticians  work 
across  service  boundaries,  i.e.,  joint  service  acquisitions,  it  is  the  one  they  normally  use.  The  sequence 
in  which  the  elements  are  listed  should  not  be  interpreted  as  establishing  a  priority. 

a.  Design  influence  and  integration  to  include  logistic-related  reliability  and  maintainability 

b.  Maintenance  concept  and  plan 

c.  Supply  support 

d.  Support  equipment  and  test,  measurement,  and  diagnostic  equipment 

e.  Training  and  training  devices 

f.  Manpower  and  personnel 

g.  Computer  resources  support 

h.  Packaging,  handling,  and  storage 

i.  Technical  data 

j.  Transportation  and  transportability 

h.  Facilities 

i.  Standardization  and  interoperability 

13.8  LOGISTIC  SUPPORT  ANALYSIS  (LSA) 

The  logistic  support  analysis  is  divided  into  3  sets,  5  task  sections,  15  tasks,  and  numerous  subtasks. 
Many  of  the  tasks  and  subtasks  will  not  be  new  because  they  have  been  applied  to  projects  for  years. 
In  fact,  they  are  considered  normal  engineering  practice.  The  major  change  has  been  to  formalize  them 
to  ensure  logistic  considerations. 


Relationship  of  LSA  sets.  Task  Sections,  and  Tasks 

Manage 

100  Program  Planning  and  Control 

101  Development  of  an  early  logistic  support  analysis  strategy 

102  Logistic  support  analysis  plan  and/or  integrated  logistic  support  plan 

103  Program  and  design  reviews 
Analysis  and  Synthesis 

200  Mission  and  Support  Systems  Definition 

201  Use  study 

202  Mission  hardware,  software,  and  support  systems  standardization 

203  Comparative  analysis 

204  Technological  opportunities 

205  Supportability  and  supportability-related  design  factors 
300  Preparation  and  Evaluation  of  Alternatives 

301  Function  requirements  definition 

302  Support  system  alternatives 

303  Evaluation  of  alternatives  and  trade-off  analysis 

400  Determination  of  Logistic  Support  Resource  Requirements 

401  Task  analysis 

402  Early  deployment  analysis 

403  Postproduction  support  analysis 
Test  and  Correct 

500  Supportability  Assessment 
501  Supportability  test,  evaluation,  and  verification 

13.9  LSA  TASK  103 

From  the  list  above  let’s  examine  MIL-STD-1388-1A  LSA  Task  Number  103  in  a  little  more  detail. 

LSA  TASK  103 
Program  and  Design  Reviews 

Purpose:  This  task  provides  for  timely  LSA  program  participation  in  the  official  review  and  control 
of  design  information;  the  scheduling  of  detailed  LSA  program  reviews;  and  logistic  risk  assessments 
at  program  reviews.  It  also  ensures  that  all  pertinent  aspects  of  the  LSA  program  are  addressed 
as  an  integral  part  of  all  formal  program  and  design  review;,. 


Required  For:  These  procedures  for  the  review  of  design  information  from  a  support  standpoint  within 
the  performing  activity  provide  logisticians  a  mechanism  for  accomplishing  design  influence  and 
tradeoffs.  LSA  program  reviews  aid  in  monitoring  the  overall  progress,  quality,  and  consistency 
of  the  LSA  effort. 


When  required:  Program  and  design  reviews  are  generally  initiated  during  the  concept  exploration 
phase  and  are  scheduled  periodically  throughout  subsequent  phases. 

Responsibility:  Initially  the  SYSCOM  program  office  (with  assistance  from  the  TDA,  if  one  has  been 
designated)  is  responsible  for  Task  103.  During  the  demonstration  and  validation  and  subsequent 
phases,  the  TDA  assumes  responsibility  for  this  task. 

Associated  Subtasks: 


Program  reviews 


Design  reviews 


Cost 

Schedule 

Performance 

Documentation 

Supportability  risk 

Assessment 


Interfaces 

Specifications 

LSA 

Supportability 
Risk  assessment 


LSA  Reviews 


Task  results 
Data  exchange 
Test  results 
Recommendations 


13.10  DETECTING  TRENDS 

It  should  be  noted  that  the  logistician  is  not  only  interested  in  what  is  happening,  but  what  may 
or  could  happen  as  well.  By  detecting  trends  early  enough  he  has  time  to  compensate  for  changes  and, 
if  necessary,  make  major  revision  of  long  range  plans. 

Many  situations  that  occur  within  an  acquisition  program  cause  that  program  to  be  delayed.  It 
is  not  uncommon  for  a  Navy  initial  operating  date  (IOC)  to  slip  3  to  5  years  beyond  the  original  estimate. 
The  logistician  is  the  individual  that  should  be  aware  of  the  impacts  of  acceleration  and  slippage  to 
the  acquisition  and  should  advise  the  program  management. 

For  example,  if  the  IOC  slips  during  the  full-scale  development  phase  what  is  the  impact  on  the 
funds  programmed  by  NAVSUPSYSCOM  and  the  Ships  Parts  Control  Center  for  the  acquisition  of 
spare  parts  to  support  the  acquisition  via  their  funding  cycle?  The  magnitude  of  dollars  runs  into  tens 
of  millions.  What  courses  of  action  are  open  to  the  supply  system  to  handle  the  sum  of  monev  which 
can’t  be  held  or  spent? 

It  is  for  situations  like  this  that  very  close  coordination  by  the  logistician  is  required  for  elements 
outside  of  the  main  stream  of  design  and  conventional  systems  engineering. 
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13.11  WHAT  DO  I  DO? 


Figure  13.2,  LSA  activities  in  the  acquisition  cycle,  provides  very  specific  guidance  as  to  the  type 
of  activities  which  are  normally  required. 

It  will  be  noted  that  the  logistical  activity  starts  even  before  the  program  initiation!  During  this 
period,  which  is  normally  taken  care  of  by  the  SYSCOMs,  definition  of  constraints  and  objectives 
is  established.  It  is  within  the  realm  of  possibility  that  a  Navy  laboratory  will  be  called  upon  to  assist 
in  this  function. 

The  bottom  line  of  Figure  13.2  indicates  the  types  of  funds  and  when  they  are  obligated.  Note 
that  the  typical  funding  for  logistic  considerations  ranges  from  6.1  (Basic  Research)  to  6.5  (RDT&E 
Management  and  Support).  Other  funds  included  are  procurement,  operations  and  maintenance,  military 
construction  (MILCON),  training,  etc.  All  of  these  funds  have  a  5-year  or  longer  cycle. 

Using  MILCON  funds  as  an  example,  it  would  be  prudent  to  consider  facilities  as  an  8-year 
requirement.  This  is  to  allow  the  definition  as  to  what  is  required,  where  it  will  be,  what  will  it  look 
like,  are  architectural  drawings  required,  site  surveys,  impact  studies,  can  the  requirements  be  combined 
with  the  potential  sites  other  plans,  and  what  special  requirements  are  needed.  These  questions  are 
only  a  few  of  those  which  must  be  answered  prior  to  the  start  of  the  funding  cycle. 

Assuming  that  the  planning  has  been  accomplished  to  determine  what  is  required  and  how  much 
it  will  cost,  consideration  must  be  given  to  the  beneficial  occupancy  date  or  that  date  when  the  facilities 
may  actually  be  used  by  the  program.  All  of  these  elements  must  go  together. 

Failure  to  consider  the  time  factor  can  disrupt  the  entire  system.  It  can  lead  to  facilities  requirements 
not  being  met  when  they  are  required  or,  if  a  program  has  a  very  high  priority,  reprogramming  of 
MILCON  funds.  Reprogramming  of  funds  can  eliminate  needed  MILCON  dollars  from  other  programs 
to  meet  unplanned  requirements.  The  adequate  planning  of  MILCON  requirements  is  fundamental, 
yet  often  overlooked. 


13.12  WHAT  HELP  IS  AVAILABLE? 

Code  936  (Computer  Integrated  Engineering),  Naval  Ocean  Systems  Center,  has  developed  a 
computer  program  that  should  be  of  major  advantage  to  program  management.  The  program  is  called 
computer-aided  logistic  support  analysis  (CALSA).  A  brief  discussion  of  the  program  follows. 

CALSA 

a.  Government-owned  computer  program  developed  by  the  NAVOCEANSYSCEN 

b.  User  friendly 

c.  Easily  tailorable  to  meet  program  requirements 

d.  Real  time 

e.  Designed  to  be  a  LIFE-CYCLE  program 

f  Being  used  by  the  Navy  on  one  program  at  the  Center  and  the  Air  Force  at  several  locations. 

g.  Has  DoD,  OPNAV,  NAVAIR,  NAVSEA,  NAVSUP,  Air  Force  Systems  Acquisition  Logistics 
Command,  Air  Force  Electronics  Command,  Army  Air  Defense  Artillery,  Army  Training  and 
Doctrine  Command,  Defense 
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Figure  13.2  LSA  activities  in  the  acquisition  cycle. 


h.  Systems  Management  College,  and  Army  Logistics  Management  Center  level  interest. 

i.  Is  under  evaluation  for  validation  by  the  Army 

j.  Is  a  planned  testbed  system  for  the  joint  logistics  Commander’s  reliability  and  maintainability 
in  computer-aided  design  demonstrations. 

Already  several  testbed  elements  have  been  applied  interactively  to  CALSA  and  will  be  widely 
available  shortly. 

Hardware  and  manpower  analysis  (HARDMAN)  (Navy  testbed  project) 

Navy  timely  spares  provisioning  (TSP)  (Navy  testbed  project) 

DoD  replenishment  parts  breakout  program  (FARS  Supplement  6)  (Testbed  project) 

Interactive  wholesale  and  retail  supply  systems  analysis  models  and  algorithms  under  development 
in  conjunction  with  NAVSUPSYSCOM  and  Ships  Parts  Control  Center  (testbed  project) 

Planned  onsite  availability  of  various  interactive  computer-aided  engineering  and  logistics  tools 

The  reader  should  be  aware  that  computer-aided  engineering  and  logistics  support  is  available  from 
Code  936,  Computer  Integrated  Engineering.  If  we  cannot  assist  you,  we  can  direct  you  to  someone 
who  can. 
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SECTION  14 

SOFTWARE  PRODUCT  ASSURANCE 
C.  Gladson,  Code  9201 


14.1  INTRODUCTION 


14.1.1  References 


See  subsection  14.2.2.1. 


14.1.2  Outline 


Introduction 

References 

Outline 

Summary 

Project  Management 
Scope 

Government  Documents 
Definitions 

Allocated  Baseline 
Authentication 
Baseline 
Certification 

Computer  Data  Definition 
Computer  Software  (or  Software) 

Computer  Software  Components  (CSC) 

Computer  Software  Configuration  Item  (CSCI) 

Computer  Software  Documentation 

Computer  Software  Quality  (or  Software  Quality) 

Configuration  Identification 

Configuration  Item 

Developmental  Configuration 

Firmware 

Format  Test 

Functional  Baseline 

Hardware  Configuration  Item  (HWCI) 

Informal  Test 
Modular 
Product  Baseline 

Software  Development  Library  (SDL) 

Top-down 

Unit 

General  Requirements 

Computer  Software  Organization 
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Software  Quality 
Subcontractor  Control 

Nondeliverable  Software,  Firmware,  and  Hardware 
Firmware 

Development  Process 
Concept  Exploration  Phase 
Demonstration  and  Validation  Phase 
Software  Requirements  Phase 
Preliminary  Design  Phase 
Detailed  Design  Phase 
Coding  and  Unit  Test  Phase 

Computer  Software  Components  (CSC)  Integration  and  Testing  Phase 
CSCI  Testing  Phase 
Software  Quality  Evaluation 
Purpose 

Internal  Reviews 

Internal  Review — Software  Requirements  Analysis 

Internal  Review— Preliminary  Design 

Internal  Review— Detailed  Design 

Internal  Review— Code  and  Unit  Testing 

Internal  Review — CSC  Integration  and  Testing 

Internal  Review— CSCI  Testing 

Evaluation 

Software  Project  Planning  and  Control 
14.1.3  Summary 

Software  development  begins  with  the  identification  of  the  need  for  a  computer  software  product 
and  ends  with  the  successful  operation  of  the  developed  software  in  the  user’s  environment.  This  section 
describes  the  Department  of  Defense’s  structured  approach  to  the  organization  of  the  activities  required 
throughout  the  development  cycle 

The  simplest  analysis  of  the  software  development  process  yields  a  three-phase  approach  (Figure  14.1). 
These  three  steps,  while  common  to  the  development  and  use  of  all  computer  programs,  are  independent 
of  the  size,  complexity,  or  application.  In  fact,  these  steps  may  be  all  that  are  required  if  the  program 
is  very  small  and  used  exclusively  by  the  implementer.  However,  a  software  development  plan  (SDP) 
designed  around  these  three  steps  cannot  succeed  for  larger  software  development  projects. 

The  major  problem  with  this  approach  is  the  lack  of  intermediate,  measurable  milestones  to  provide 
checkpoints  for  the  development  process.  To  introduce  meaningful  checkpoints  would  produce  the 
software  development  methodology  shown  in  Figure  14.2.  Each  phase  or  step  ends  with  a  measurable 
milestone  (e.g.,  complete  software  specification,  preliminary  design,  detailed  design).  Furthermore, 
each  phase  of  the  process  will  require  iterations  with  adjacent  phases  and  to  a  lesser  degree  iteration 
with  phases  further  back  in  the  process.  This  provides  a  fallback  position  allowing  effective  use  of 
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Figure  14  1  The  three-phase  software  development  process 
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earlier  work  in  the  development  process.  The  definition,  refinement,  and  formalization  of  products 
and  the  monitoring  techniques  for  these  processes  make  up  the  project  activities  This  software 
development  process  is  divided  into  six  phases: 

a.  System  requirements  analysis 

b.  Detailed  design 

c.  Coding 

d.  Unit  testing 

e.  Integration  testing 

f.  Computer  software  configuration  item  (CSCI)  certification  testing.  Tasks  performed  within  each 
phase  of  the  software  development  process  produce  documents  required  to  control  and  monitor 
the  software  design  and  to  produce  coded  programs,  verified  and  delivered  to  the  user.  Technical 
reviews  are  used  to  formalize  the  development  control  process  and  to  validate  budget  and  schedule 
reports. 


14.2  PROJECT  MANAGEMENT 

Early  in  the  project  definition  phase  for  internal  projects  or  during  preproposal  activities  for 
competitive  procurements,  the  selected  project  manager  needs  to  take  the  lead  in  establishing  the  project 
starting  point.  This  task  includes  identifying  high-risk  areas  and  establishing  budgets,  schedules,  staffing 
plan,  task  allocations,  support  organizational  requirements,  project  interface  techniques,  subcontractor 
requirements,  and  the  project/customer  interface  approach.  The  planning  for  these  activities  is  required 
to  be  included  in  the  project  management  plan. 

The  project  management  plan  (PMP)  defines  the  project  starting  point.  This  plan  also  provides 
the  definitions  of  what  will  be  done,  how,  by  whom,  on  what  schedule,  and  which  development  and 
management  tools  and  techniques  will  be  used.  The  project  manager  controls  the  software  development 
activities  on  the  basis  of  plans  prepared  at  the  start  of  the  project  and  updated  as  required.  There  are 
several  management  tools  that  can  assist  the  project  manager  in  successfully  completing  the  software 
development  project,  e.g.,  schedules,  WBS,  documentation,  configuration  control,  standards  and 
conventions,  and  subcontracting.  The  PMP  should  include  sections  for  each  of  these  areas.  The  PMP 
should  also  describe  the  intended  use  of  each  of  these  management  tools  to  support  the  project  manager’s 
task  of  planning,  progress  analysis,  problem  resolution,  and  project  coordination. 

14.2.1  Scope  (DOD-STD-2167) 

14.2.1.1  Purpose.  DoD-STD-2167  establishes  requirements  to  be  applied  during  the  development 
and  acquisition  of  mission  critical  computer  systems  (MCCS). 

14.2.1.2  Application.  DoD-STD-2167  applies  to 

a.  Deliverable  software  designated  as  a  computer  software  configuration  item  (CSCI) 
b  Development  as  part  of  a  hardware  configuration  item  (HWCI) 

c.  Nondeliverable  development  and  test  software 

d.  Deliverable  unmodified  commercial  and  reusable  software 

e.  Modified  commercial,  GFI,  and  reusable  software. 


14.2.1.3  Software  Development  b>  Government  Agencies.  The  provisions  of  Mil  -STD-2l<  '  api  v 
to  government  agencies  acting  as  software  developers.  In  this  case,  the  term  “contractor "  refers  'o 
the  government  agency  that  is  developing  the  software.  Any  contractor  of  that  government  agones 
is  classified  as  a  subcontractor. 

14.2.1.4  Tailoring  MIL-STD-2167.  The  contracting  agency  will  tailor  DoD-STD-2167  to  require  only 
what  is  needed  for  each  individual  acquisition.  Guidelines  for  applying  this  standard  are  provided  in 
Appendix  O  of  DoD-STD-2167. 

14.2.2  Government  Documents 

14.2.2.1  Specifications,  Standards,  and  Handbooks.  Unless  otherwise  specified,  the  following 
specifications,  standards,  and  handbooks  of  the  issue  listed  in  that  issue  of  the  Department  of  Defense 
Index  of  Specifications  and  Standards  (DoDISS)  specified  in  the  solicitation  form  a  part  of  DoD- 
STD-2167  to  the  extent  specified  herein. 

STANDARDS,  MILITARY 

DoD-STD-480  Configuration  Control — Engineering  Changes,  Deviations,  and  Waivers 

MIL-STD-481  Configuration  Control — Engineering  Changes,  Deviations  and  Waivers  (Short 

Form) 

MIL-STD-483  Configuration  Management  Practices  for  Systems,  Equipment,  Munitions,  and 
Computer  Software 

M1L-STD-490  specification  Practices 

MIL-STD-881  Work  Breakdown  Structures  for  Defense  Material  Items 

MIL-STD-1521  Technical  Reviews  and  Audits  for  Systems,  Equipments,  and  Computer  Software 

MIL-STD-1535  Supplier  Quality  Assurance  Program  Requirements 

14.2.2.2  Other  Government  Documents,  Drawings,  and  Publications.  None.  (Copies  of  specifications, 
standards,  handbooks,  drawings,  and  publications  required  by  contractors  in  connection  with  specific 
acquisition  functions  should  be  obtained  from  the  contracting  agency  or  as  directed  by  the  contracting 
officer.) 

14.2.2.3  Other  Publications.  None. 

14.2.2.4  Order  of  Precedence.  In  the  event  of  a  conflict  between  the  text  of  this  standard  and  the 
reterences  cited  herein,  the  text  of  this  standard  shall  take  precedence. 

14.3  DEFINITIONS 
14.3.1  Allocated  Baseline 

The  initial  approved  allocated  configuration  identification  as  specified  in  Poll  STD-480 


14.3.2  Authentication 


The  procedure  (essentially  approval)  used  by  the  government  in  verifying  that  specification  content 
is  acceptable.  Authentication  does  not  imply  acceptance  or  responsibility  by  the  government  for  the 
specified  item  to  perform  successfully. 

14.3.3  Baseline 

A  configuration  identification  document  or  a  set  of  such  documents  (regardless  of  media)  tormally 
designated  and  fixed  at  a  specific  time  during  a  configuration  item’s  life  cycle.  Baselines,  plus  approved 
changes  from  those  baselines,  constitute  the  current  configuration  identification. 

14.3.4  Certification 

A  process,  which  may  be  incremental,  by  which  a  contractor  provides  evidence  to  the  contracting 
agency  that  a  product  meets  contractual  or  otherwise  specified  requirements. 

14.3.5  Computer  Data  Definition 

A  statement  of  the  characteristics  of  basic  elements  of  information  operated  upon  by  hardware 
in  responding  to  computer  instructions.  These  characteristics  may  include,  but  are  not  limited  to,  type, 
range,  structure,  and  value. 

14  V6  Computer  Software  (or  Software) 

A  combination  of  associated  computer  instructions  and  computer  data  definitions  required  to  enable 
the  computer  hardware  to  perform  computational  or  control  functions. 

14.3.7  Computer  Software  Components  (CSC) 

A  functional  or  logically  distinct  part  of  a  computer  software  configuration  item  C  omputer  software 
components  may  be  top-level  or  lower-level. 

14.3.8  Computer  Software  Configuration  Item  (CSCI) 

See  Configuration  Item  (14.3.12). 

14.3.9  Computer  Software  Documentation 

Technical  data  or  information,  including  computer  listings  and  printouts,  which  document  the 
requirements,  design,  or  details  of  computer  software;  explain  the  capabilities  and  limitations  of  the 
software:  or  provide  operating  instructions  for  using  or  supporting  cor.puter  software  during  the 
software’s  operational  life. 


14.3.10  Computer  Software  Quality  (or  Software  Quality) 


The  degree  to  which  the  attributes  of  the  software  enable  it  to  perform  its  specified  end  item  use. 
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14.3.11  Configuration  Identification 


The  current  approved  or  conditionally  approved  technical  documentation  for  a  configuration  item 
as  set  forth  in  specifications,  drawings,  and  associated  lists,  and  documents  referenced  therein. 


14.3.12  Configuration  Item 

Hardware  or  software,  or  an  aggregation  of  both,  which  is  designated  by  the  contracting  agency 
for  configuration  management. 

14.3.13  Developmental  Configuration 

The  contractor’s  software  and  associated  technical  documentation  that  define  the  evolving 
configuration  of  CSCI  during  development.  It  is  under  the  development  contractor’s  configuration 
control  and  describes  the  software  configuration  of  the  design,  coding,  and  testing  effort.  Any  item 
in  the  development  configuration  may  be  stored  on  electronic  media. 


14.3.14  Firmware 

The  combination  of  a  hardware  device  and  computer  instructions  or  computer  data  that  reside 
as  read-only  software  on  the  hardware  device.  The  software  cannot  be  readily  modified  under  program 
control.  The  definition  also  applies  to  read-only  digital  data  that  may  be  used  by  electronic  devices 
other  than  digital  computers. 

14.3.15  Format  Test 

A  test  conducted  in  accordance  with  test  plans  and  procedures  approved  by  the  contracting  agency 
and  witnessed  by  an  authorized  contracting  agency  representative  to  show  that  the  software  satisfies 
a  specified  requirement. 

14.3.16  Functional  Baseline 

The  initial  approved  functional  configuration  identification  as  specified  in  DoD-STD-480. 

14.3.17  Hardware  Configuration  Item  (HWCI) 

See  Configuration  Item  (14.3.12). 


14.3.18  Informal  Test 

•\nv  test  which  docs  not  meet  all  the  requirements  of  a  formal  test. 

14.3.19  Modular 

Pei  Mining  to  software  that  is  organized  into  limited  aggregates  of  data  and  contiguous  codes  that 
peCorm  identifiable  functions. 


14.3.20  Product  Baseline 


The  initial  approved  product  configuration  identification  as  specified  in  DoD-STD-480. 

14.3.21  Software  Development  Library  (SDL) 

A  controlled  collection  of  software,  documentation,  and  associated  tools  and  procedures  used  to 
facilitate  the  orderly  development  and  subsequent  support  of  software.  A  software  development  library 
provides  storage  of  and  controlled  access  to  software  and  documentation  in  both  human-readable  and 
machine-readable  form.  The  library  may  also  contain  management  data  pertinent  to  the  software 
development  project. 

14.3.22  Top-down 

Pertaining  to  an  approach  that  starts  with  the  highest  level  of  a  hierarchy  and  proceeds  through 
progressively  lower  levels.  For  example,  top-down  design,  top-down  coding,  top-down  testing. 

14.3.23  Unit 

The  smallest  logical  entity  specified  in  the  detailed  design  which  completely  describes  a  single  function 
in  sufficient  detail  to  allow  implementing  code  to  be  produced  and  tested  independently  of  other  units. 
Units  are  the  actual  physical  entities  implemented  in  code. 


14.4  GENERAL  REQUIREMENTS 

The  contractor/developer  shall  implement  a  software  development  cycle  that  includes  the  following 
six  phases: 

a.  Software  requirements  analysis 

b.  Preliminary  design 

c.  Detailed  design 

d.  Coding  and  unit  testing 

e.  CSC  integration 

f.  CSC1  testing. 


14.4.1  Computer  Software  Organization 

Computer  software  developed  in  accordance  with  DoD-STD-1679  shall  be  organized  as  one  or 
more  CSCIs  or  other  types  of  software.  Each  CSCI  is  part  of  a  system,  system  segment,  or  prime  item 
and  shall  consist  of  one  or  more  top-level  computer  software  components  (TLCSCs).  Each  TLCSC 
shall  consist  of  lower-level  computer  software  components  (LLCSCs)  or  units.  TLCSCs  and  LLCSCs 
are  logical  groupings.  Units  are  the  smallest  logical  entities,  and  the  actual  physical  entities  implemented 
in  code.  The  static  structure  of  CSCIs,  TLCSCs,  LLCSCs,  and  units  shall  form  a  hierarchical  structure 
and  shall  uniquely  identify  all  CSCIs,  TLCSCs,  LLCSCs,  and  units.  The  partitioning  of  the  components 
and  units  may  be  based  on  functional  requirements,  data  flow  requirements,  or  other  design 
considerations. 


14.4.2  Software  Qualify 


The  contractor  developer  shall  plan  and  implement  the  software  development  project  wi:h  the 
objective  of  building  in  quality.  To  achieve  this  quality  the  contractor  shall 

a.  Establish  and  maintain  a  complete  set  of  requirements. 

b.  Establish  and  implement  a  complete  process  for  developing  the  software  (SDP,  SCMP,  and 

SSPM). 

c.  Establish  and  maintain  a  software  quality  evaluation  process  (SDEP). 

14.4.3  Subcontractor  Control 

When  NOSC  is  the  software  development  agency,  NOSC  plays  the  role  of  a  prime  contractor. 
A  prime  contractor  shall  ensure  that  all  subcontractors  developing  software  and  documentation  comply 
with  subcontracting  requirements. 

14.4.4  Nondeliverable  Software,  Firmware,  and  Hardware 

The  contractor/developer  shall  describe  in  the  SDP  the  controls  to  be  imposed  on  all  nondeliverable 
software,  firmware,  and  hardware  used  in  the  development  and  acquisition  of  deliverable  software. 
As  a  minimum,  the  contractor/developer  shall  describe  the  provisions  for 

a.  Modifications 

b.  Documentation 

c.  Configuration  management 

d  Design  and  coding  standards 

e.  Testing 

f.  Quality  evaluation 

g.  Certification. 

14.4.5  Firmware 

The  application  of  DoD-STD-2167  to  firmware  depends  on  whether  the  firmware  is  designated 
as  a  CSCI  or  as  part  of  a  HWCI.  If  the  software  to  be  implemented  in  firmware  is  considered  part 
of  the  HWCI,  the  contractor/developer  shall  identify  the  applicable  requirements  in  the  SDP.  These 
requirements  are  subject  to  the  contracting  agency  approval/ disapproval. 

14.5  DEVELOPMENT  PROCESS 

14.5.1  Concept  Exploration  Phase 

14.5.1.1  Purpose.  The  objectives  of  the  concept  exploration  phase  are  to  explore  svstem  concepts 
and  to  determine  the  feasibility  of  using  computet  resources  to  satisfy  operational  needs.  This  phase 
includes 
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a.  Defining  system  level  requirements 

b.  Analyzing  development  concepts 

c.  Analyzing  alternative  allocation  of  system  requirements 

d.  Defining  intersystem  interfaces 

e.  Developing  initial  planning  documents. 

14.5.1.2  Products.  The  following  engineering  products  are  developed  during  this  phase: 

a.  Preliminary  system/segment  specification  (SSS)  DI-MCCR-80008 

b.  Preliminary  test  and  evaluation  master  plan  (TEMP) 

c.  Preliminary  computer  resource  life  cycle  management  plan 
(CRLCMP) 

14.5.1.3  System  Requirements  Review  (SRR)  and  Baseline.  A  system  requirements  review  will  be 
held  to  evaluate  the  adequacy  of  system  requirements  contained  in  the  draft  SSS  in  meeting  the  stated 
operational  needs. 

The  authenticated  SSS  establishes  the  functional  baseline.  However,  the  SSS  is  normally 
authenticated  at  the  system  design  review. 

14.5.1.4  Activities,  Plans,  and  Controls.  After  a  need  for  a  new  mission  capability  has  been  identified 
and  validated,  a  program  will  be  initiated  to  explore  alternative  system  concepts.  Concept  exploration 
may  be  directed  toward  refining  proposed  solutions  or  developing  new  concepts. 

Exp'  tratory  activities  may  include: 

a.  System  engineering  studies 

b.  Feasibility  studies 

c.  Trade-off  studies 

d.  Risk  assessments 

e.  Requirements  definition 

f.  Computer  resource  use  studies 

g.  Operational  concept  analysis 

h.  Support  concept  studies 

i.  Test  and  evaluation  planning 

j.  Initial  software  quality  planning 

k.  Independent  verification  and  validation  planning. 

14.5.1.5  Management  Documents.  The  following  management  documents  are  normally  developed 
during  this  phase; 

a.  Initial  software  quality  evaluation  plan  (SQEP)  DI-MCCR- 

b.  Initial  computer  resource  life  cycle  management  plan  (CRLCM) 
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14.5.1.6  Quality  Factors.  Software  quality  planning  will  begin  duitng  this  phase  equality  factors 
will  be  defined  and  the  overall  soltware  evaluation  process  for  the  software  development  cycle  will 
be  established  to  the  maximum  extent  possible.  The  planning  will  include  a  determination  of  the  level 
of  IV&V  (independent  validation  and  verification)  to  be  used  during  subsequent  phases.  Achieving 
software  quality  requires  that  the  quality  be  built  in  from  the  start  and  that  it  be  evaluated  throughout 
the  software  development  cycle. 

14.5.1.7  Qualification  Requirements.  See  Demonstration  and  Validation  Phase  (14.5.2). 

14.5.2  Demonstration  and  Validation  Phase 

14.5.2. 1  Purpose.  The  objectives  of  the  demonstration  and  validation  phase  are  to  validate  system 
requirements  and  to  demonstrate  that  the  system,  including  its  computer  resources,  is  suitable  for 
engineering  development.  During  this  phase,  system  requirements  are  allocated  and  computer  resource 
life  cycle  planning  is  completed. 

14.5.2.2  Products.  The  following  engineering  products  will  be  developed  m  final  or  preliminary  form 
during  this  phase: 

a.  System/  segment  specification  (SSS)  (finalize)  DI-.V1CCR-80008 

b.  Computer  resource  life-cycle  management  plan  (CRLCMP)  (finalize) 

c.  Test  and  evaluation  master  plan  (TEMP)  (finalize) 

d.  Preliminary  operational  concept  document  (OCDP)  DI-MCCR-80023 

e.  Preliminary  software  requirements  specification  (SRS)  Df-MCCR-80025 

f.  Preliminary  interface  requirements  specifications  (IRS)  Dl  MCCR-80026 
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14.5.2.3  System  Design  Review  (SDR)  and  Baseline.  The  purpose  of  the  SDR  is  to  formally  assess 
the  allocated  system  requirements  before  proceeding  with  the  software  requirements  analysis  and  the 
preliminary  design  of  the  software  and  hardware  The  SDR  will  evaluate  the  optimization,  traceability, 
completeness,  and  risk  associated  with  the  allocated  requirements.  A  successful  SDR  will  be  predicted 
on  the  determination  that  the  SSS  is  an  adequate  basis  for  developing  hardware  and  software 
configuration  items.  The  functional  baseline  defines  the  system  as  it  enters  the  full-scale  development 
(FSD)  phase.  If  the  SSS  has  not  previously  been  authenticated  it  will  be  authenticated  following  the 
successful  completion  of  the  SDR  and  will  establish  the  functional  baseline.  The  functional  baseline, 
established  by  the  authenticated  SSS,  will  be  under  government  configuration  control. 

14.5.2.4  Activities,  Plans,  and  Controls.  System  requirements  will  be  completed  and  defined  for 
each  HWCI  and  CSCI.  The  contracting  agency  will  plan  for 

a.  System  engineering  studies 

Feasibility  studies 

c  Trade-off  and  optimization  studies 
d  Risk  management 
e.  Definition  of  the  system  requirements 
I.  Validation  of  requirements 
g  Software  support 
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h.  Interface  definition 

i.  Prototype  computer  resources. 

The  continuous  activities  are 

a.  Computer  resource  life  management  planning 

b.  Computer  resource  working  group  (CRWG)  support 

c.  Configuration  management 

d.  Software  quality  evaluation 

e.  Test  and  evaluation  planning 

f.  Independent  verification  and  validation  support 

g.  Operational  concepts  refinement. 

14.5.2.5  Management  Documents.  The  contractor/developer  shall  develop  and/or  update  and 
establish  internal  control  over: 


a.  Software  development  plan  (SDP) 

b.  Software  configuration  management  plan  (SCMP) 

c.  Software  quality  evaluation  plan  (SQEP) 

d.  Software  standards  and  procedures  (SSPM)  manual 


DI-MCCR-80030 
DI-MCCR-80009 
DI-MCCR-80010 
DI-MCCR-8001 1 


14.5.2.6  Quality  Factors.  The  quality  factors  pertaining  to  system  quality  will  be  soecified.  Typical 

q'uaL;\  factors  are 

a.  Reliability 

b.  Modificability 

c.  Maintainability 

d.  Flexibility 

e.  Availability 

f.  Portability 

g.  Efficiency. 

14.5.2.7  Qualification  Requirements.  System  level  qualification  requirements  shall  be  specified  during 
this  phase.  Typically,  the  following  information  is  required: 

a.  Qualification  methods 

b.  Philosophy  of  testing 

c.  Location  of  testing 

d.  Responsibility  for  tests 

e.  Test  levels 

f.  Formal  test 

g.  Formal  test  constraints. 
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14.5.3  Software  Requirements  Phase 

14.5.3.1  Purpose.  The  purpose  of  the  software  requirements  is  to  define  and  document  the  functional, 
performance,  interface,  and  qualification  requirements  for  each  computer  software  configuration  item 
(CSCI).  The  requirements  will  be  derived  from  the  system  requirements  as  defined  in  the  system/segment 
specification  (SSS). 


14.5.3.2  Products.  The  engineering  products  produced  during  this  phase  are 

a.  Software  requirements  specification  (SRS)  DI-MCCR-80025 

b.  Interface  requirements  specification  (IRS)  DI-MCCR-80026 

c.  Completed  operational  concept  document  (OCD)  DI-MCCR-80023 


14.5.3.3  Formal  Design  Reviews  and  Baseline.  At  the  conclusion  of  the  software  requirements  phase, 
a  software  specification  review  (SSR)  is  held.  Upon  completion  of  the  SSR  and  when  authenticated 
by  the  contracting  agency,  the  SRS  and  IRSs  will  establish  the  allocated  baseline  for  each  CSCI.  See 
MIL-STD-483,  1521B,  and  490  regarding  the  baseline  process. 


14.5.3.4  Activities,  Plans,  and  Controls.  The  contractor’s/developer’s  plans,  controls,  and  activities 
for  software  development  shall  include: 

a.  Resources  and  organization 

b.  Schedules  and  milestones 

c.  Standards  and  procedures 

d.  Configuration  management 

e.  Quality  evaluation 

f.  Data  rights 

g.  Nondeliverable  software 

h.  Software  that  is  part  of  a  HWCI 

i.  Interface  management  between  contractors. 


14.5.3.5  Management  Documents, 
internal  control  over 


The  contractor/developer  shall  develop  or  finalize  and  establish 


a.  Software  development  plan  (SDP) 

b.  Software  standards  and  procedures  manual  (SSPM) 

c.  Software  configuration  management  plan  (SCMP) 

d.  Software  quality  evaluation  plan  (SDF.P) 


DI-MCCR-80030 
DI-MCCR-8001 1 
DI-MCCR-80009 
Pl-MCCR-80010 


14.5.3.6  Quality  Factors.  The  quality  factor  requirements  applicable  to  each  CSCI  shall  be  specified, 
defined,  and  included  in  the  software  requirements  specification.  A  candidate  set  of  quality  factors 
is  noted  below: 


a.  Correctness 

b.  Reliabilitv 


c.  Efficiency 

d.  Integrity 

e.  Usability 

f.  Maintainability 

g.  Testability 

h.  Portability 

i.  Reusability 

j.  Interoperability. 

In  addition  to  establishing  quality  factors,  a  traceability  table  mapping  software  requirements  to 
corresponding  system  requirements  shall  be  developed  and  maintained  current  and  correct. 

14.5.3.7  CSCI  Qualification/Quality  Evaluation  Requirements.  During  this  phase,  the 
contractor/developer  shall  establish  the  qualification  methods  which  will  be  used  to  show  that  the 
requirements  of  the  CSCI  have  been  satisfied.  Typical  qualification  methods  are 

a.  Demonstration 

b.  Testing 

c.  Analysis 

d.  Inspection. 

The  contractor/developer  is  required  to  monitor  the  software  development  effort  for  consistency 
with  the  SDP,  SCCMP,  SCMP,  and  SQEP  and  to  notify  the  contracting  agency  01  proposed  changes 
to  these  documents.  The  proposed  changes  are  subject  to  disapproval  by  the  contracting  agency.  In 
addition,  the  contractor/developer  shall  notify  the  contracting  agency  of  any  actions  or  procedures 
that  deviate  from  the  SDP,  SSPM,  SCMP,  or  SQEP. 

14.5.4  Preliminary  Design  Phase 

14.5.4.1  Purpose.  The  purpose  of  the  preliminary  design  phase  is  to  develop  a  top-level  design  for 
each  CSCI  which  completely  reflects  the  requirements  contained  in  the  SRS  and  the  IRSs.  In  addition, 
the  contractor  should  develop  lower-level  design  for  critical/high  risk  elements  of  the  CSCI. 

14.5.4.2  Products.  The  engineering  products  produced  during  this  phase  are 

a.  Software  top-level  design  specification  (STLDS)  DI-MCCR-80012 

b.  Software  test  plan  (STP)  DI-MCCR-80014 

c.  Preliminary  computer  resources  integrated  support  document 

(CRISD)  Dl-MCCR-80024 

d.  Preliminary  computer  systems  operators  manual  (CSOM)  DI-MCCR-80018 

e.  Preliminary  software  users  manual  (SUM)  DI-MCCR-80019 

f.  Preliminary  computer  systems  diagnostics  manual  DI-MCCR-80020 
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14.5.4.3  Formal  Design  Review  and  Baseline.  The  purpose  of  the  preliminary  design  review  (PDR) 
is  to  review  the  top-level  design,  the  test  plans,  and  the  preliminary  operation  and  support  documents 
with  the  contracting  agency  and  to  demonstrate  that 

a.  The  top-level  design  satisfies  the  software  requirement  allocated  from  the  software  requirements 
specifications  and  the  system  requirements. 

b.  The  test  plan  establishes  adequate  test  criteria  to  qualify  each  CSCI  and  addresses  all  specified 
requirements. 

c.  The  preliminary  versions  of  the  CSOM,  SUM,  CSDM,  and  CR1SD,  will,  in  final  form, 
adequately  address  the  operation  and  support  of  the  computer  system. 

d.  For  critical  lower-level  elements  being  designed  concurrently  with  the  top-level  elements,  the 
preliminary  versions  of  the  SDDD,  IDDs,  and  DBDDs  should  be  reviewed. 

Documents  produced  during  the  preliminary  design  phase  are  entered  into  the  development 
configuration  and  controlled  by  the  contractor/developer. 

14.5.4.4  Activities,  Plans,  and  Controls.  The  contractor/developer  shall  monitor  the  development 
effort  for  consistency  and  compliance  with  the 

a.  Software  development  plan 

b.  Software  configuration  management  plan 

c.  Software  quality  evaluation  plan 

d.  Software  standards  and  procedures  manual. 

During  this  phase,  the  contractor/developer  shall  establish  the  top-level  design  to  each  CSC!  by 
allocating  requirements  from  the  SRS  and  IRSs  to  the  top-level  components  of  each  CSCI.  In  defining 
each  top-level  component,  the  contractor/developer,  as  a  minimum,  shall  identify 

a.  Top-level  components  place  in  the  CSCI  structure 

b.  Functions  allocated  to  the  top-level  component 

c.  Memory  size  and  processing  time 

d.  Functional  control  and  data  flow 

e.  Known  interrupts  and  special  control  features 

f.  Global  data  shared  with  other  top-level  components 

g.  Inputs,  local  data,  interrupts,  timing  and  sequencing,  processing,  and  outputs  of  the  top-level 
component. 

Test  plans  for  both  informal  and  formal  test  shall  be  developed. 

Informal  testing  includes  unit  testing  and  integration  testing.  Information  test  documentation  does 
not  require  government  approval.  However,  it  shall  be  made  available  for  government  review. 

For  unit  testing,  the  contractor/developer  shall  identify 

a.  Overall  test  requirements 

b.  Test  responsibilities 

c.  Schedule  information. 
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Formal  testing  consists  of  testing  fully  implemented  CSCl(s)  to  demonstrate  that  each  CSCI  satisfies 
its  specified  requirements.  Formal  testing  also  applies  to  top-level  components,  low-level  components, 
and  unit  testing  when  compliance  with  specified  requirements  cannot  be  demonstrated  at  the  CSCI 
level.  Formal  test  documentation  requires  government  approval. 

For  formal  test,  the  contractor/developer  shall,  as  a  minimum,  identify 

a.  Test  requirements 

b.  Test  organization,  responsibilities,  and  schedule 

c.  Classes/types  of  formal  tests 

d.  Data  recording,  reduction,  and  analysis 

e.  Purpose  of  each  formal  test. 

14.5.4.5  Management  Documents.  No  new  management  documents  developed  during  this  phase. 
Existing  management  documents  should  be  reviewed  and  updated. 

14.5.4.6  Quality  Factors.  Achieving  software  quality  requires  that  quality  be  built  in  during  the 
development  process  and  evaluated  during  each  phase.  The  appropriate  quality  factors  and  quality 
measurements  should  be  prescribed  in  the  SQEP  and  procedures.  Flow-down  of  quality  factors  from 
the  SRS  to  lower-level  activities  is  a  requirement. 

14.5.4.7  Qualification/Evaluation  Requirements.  The  contractor/developer  is  required  to  monitor 
the  software  development  effort  for  consistency  with  the  SDP,  SSPM,  SCMP,  and  SQEP  and  to  notify 
the  contracting  agency  of  proposed  changes.  The  changes  are  subject  to  disapproval  by  the  contracting 
agency.  In  addition,  the  contractor/developer  shall  notify  the  contracting  agency  of  any  actions  or 
procedures  that  deviate  from  the  SDP,  SSPM,  SCMP,  or  SQEP. 


14.5.5  Detailed  Design  Phase 

14.5.5.1  Purpose.  The  purpose  of  the  detailed  design  phase  is  to  describe  in  detail  the  structure  and 
organization  of  a  particular  CSCI  and  to  describe  the  decomposition  of  the  top-level  components  into 
low-level  components  and  units. 

14.5.5.2  Products.  The  engineering  products  produced  during  this  phase  are 

a.  Software  detailed  design  document  (SDDD)  DI-MCCR-80031 

b.  Software  test  description  (STD)  DI-MCCR-80015 

c.  Software  development  files,  informal 

d.  Informal  test  descriptions,  informal 

e  Computer  resources  integrated  support  document  (CRISD)  DI-MCCR-80024 

f.  Software  programmer’s  manual  (SPM)  DI-MCCR-80021 

g.  Interface  design  document  (IDS)  Dl-MCCR-80027 

h.  Data  base  design  document  (DBDD)  DI-MCCR-80028 
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14.5.5.3  Critical  Design  Review  (CDR)  and  Baseline.  The  purpose  of  the  CRD  is  10  review  the  detailed 
design,  test  description,  and  operation  and  support  documents  with  the  contracting  agency  and  to 
demonstrate  that 

a.  The  detailed  design  satisfies  the  requirement  of  the  SRS  and  the  IDSs. 

b.  The  SDDD,  IDDs,  and  DBDDs  refine  the  design  details  of  the  CSC1  in  a  manner  consistent 
with  the  STLDD. 

c.  The  STD  provides  adequate  test  cases  for  the  formal  test  identified  in  the  STP. 

d.  The  updated  versions  of  the  CSOM,  SUM,  and  CSDM  will,  in  final  form,  adequately  address 
the  operation  and  support  of  the  computer  system. 

e.  The  SPM,  FSM,  and  CR1SD  adequately  address  software  programming  support,  firmware 
support,  and  integrated  computer  resource  support. 

The  SDDD,  IDDs,  and  the  DBDDs  are  a  part  of  the  development  baseline. 

14.5.5.4  Activities,  Plans,  and  Controls.  The  contractor/develnper  shall  monitor  the  development 
effort  for  consistency  with  the  SDP,  SSPM,  SCMP,  and  SQEP  and  shall  notify  the  contracting  agency 
of  proposed  changes  to  these  documents.  The  contracting  agency  has  disapproval  authority.  In  addition, 
the  contracting  agency  must  authorize  any  actions  or  procedures  that  deviate  from  the  SDP,  SSPM, 
SCMP,  or  SQEP. 

The  contractor/developer  shall  establish  the  complete,  modular,  low-level  design  for  each  CSCI, 
and  be  refining  top-level  design  into  low-level  components  and  low-level  components  into  units.  Each 
unit  shall  perform  a  single  function. 

The  contractor/developer  shall 

a.  Use  top-down  design  unless  other  methodologies  are  described  in  the  SDP  and  SSPM  plans 
and  the  contracting  agency  has  reviewed  and  not  disapproved  the  SDP  and  SSPM  documents. 

b.  Employ  a  design  language, 
c  Incorporate  human  factors. 

d.  Monitor  size  and  timing  estimates. 

e.  Establish  software  development  files. 

f.  Document  engineering  analysis  and  trade-off  studies. 

g.  Identify  test  requirements  for  informal  tests. 

h.  Describe  test  cases  for  informal  test. 

i.  Describe  test  cases  for  formal  test. 

j.  Update  CSOM,  SUM,  CSDM,  and  CRISD. 

k.  Prepare  information  to  facilitate  software  and  target  computer  compatibility. 

l.  Prepare  information  necessary  to  maintain  firmware. 

m.  Conduct  internal  code,  PDL,  test,  and  design  walkthroughs  and  reviews. 

14.5.5.5  Management  Documents.  No  new  management  documents  produced  during  this  phase. 
Existing  management  documents  should  be  reviewed  and  updated. 


14.5.5.6  Quality  Factors 

Achieving  software  quality  requires  that  quality  be  built  in  during  the  development  process  and 
evaluated  during  each  phase.  The  appropriate  quality  factors  and  quality  measures  should  be  prescribed 
in  the  SQEP  and  the  quality  procedures.  Flow-down  of  quality  factors  from  the  SAS  to  lower-level 
activities  is  a  requirement. 

14.5.5.7  Qualification/Quality  Evaluation  Requirements.  The  contractor/  developer  is  required  to 
monitor  the  software  development  effort  for  consistency  with  the  SDP,  SSPM,  SCMP,  and  SQEP 
and  to  notify  the  contracting  agency  of  proposed  changes  to  these  documents.  The  changes  are  subject 
to  disapproval  by  the  contracting  agency.  In  addition,  the  contractor/developer  shall  notify  the 
contracting  agency  of  any  actions  or  procedures  that  deviate  from  the  SDP,  SSPM,  SCMP,  or  SQEP. 

14.5.6  Coding  and  Unit  Test  Phase 

14.5.6.1  Purpose.  The  objectives  of  this  phase  are  to  develop  the  code  and  demonstrate  that  the 
detailed  design  is  accurately  translated  in  code. 

14.5.6.2  Products.  The  following  engineering  products  are  developed  or  updated  during  this  phase: 

a.  Preliminary  software  test  procedure  (STPR)  DI-MCCR-80016 

b.  Source  code 

c.  Object  code 

d.  Informal  test  procedures 

e.  Informal  test  results 

f.  Update  CSOM,  SUM,  CSDM,  SPM,  and  FSM  as  required. 

14.5.6.3  Formal  Design  Review  and  Baselines.  Source  code,  object  code,  test  document,  and  software 
development  files  are  placed  under  internal  configuration  management  and  become  a  part  of  the 
development  baseline.  A  formal  design  review  is  not  held  at  this  milestone. 

14.5.6.4  Activities,  Plans,  and  Controls.  The  contractor/developer’s  activities,  plans,  and  controls 
normally  consist  of  the  following: 

a.  Top-down  coding  and  unit  testing  unless  other  methodologies  have  been  proposed  in  either 
the  SSPM  or  SDP  and  have  received  contracting  agency  approval.  Candidates  for  a  departure 
from  top-down  approach  are  critical  units,  government-furnished  software,  and  commercially 
available  software. 

b.  Use  of  coding  standards. 

c.  Use  of  the  software  development  folder  (SDF). 

d.  Unit  testing  will  be  controlled  by  the  test  plans  contained  in  the  STP  and  performed  in  accordance 
with  the  unit  test  cases  and  unit  test  procedures  contained  in  the  SDF. 

e.  Record  all  unit  test  results  in  the  unit  development  folder  (UDF). 

f.  Maintain  all  documents  in  a  current  status. 

g.  Develop  test  procedures  for  integration  test, 
h  Conduct  code  and  test  in-process  reviews. 
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14.5.6.5  Management  Documents.  The  contractor/ developer  shall 

/V\ 

a.  Update  the  SDP,  SSPM,  SCMP,  and  SEQP  as  required 

*  m 

b.  Produce  internal  review  records 

c.  Produce  updated  SDFs. 

14.5.6.6  Quality  Factors.  The  software  quality  metrics  and  measures  that  support  quality  factors 
evaluation  should  be  a  flow-down  from  higher-level  requirements.  The  SQEP  should  address  this  process. 

14.5.6.7  Quaiification/Quality  Evaluation  Requirements.  Formal  qualification  of  components  during 
this  phase  would  only  be  performed  on  those  units  which  contain  functions  that  cannot  be  qualified 
at  the  CSCI  level.  Formal  tests  conducted  during  this  phase  require: 

a.  Formal  test  procedures 

b.  Contracting  agency  approval  of  the  test  procedures 

c.  Test  performed  in  accordance  with  the  approved  test  procedures. 

The  contractor/developer  is  required  to  monitor  the  software  development  effort  for  consistency 
with  the  SDP,  SSPM,  SCMP,  and  SQEP  and  to  notify  the  contracting  agency  of  proposed  changes 
to  these  documents.  The  changes  will  be  subject  to  disapproval  by  the  contracting  agency.  In  addition, 
the  contractor/developer  shall  notify  the  contracting  agency  of  any  actions  or  procedures  that  deviate 
from  the  SDP,  SSPM,  SCMP,  or  SQEP. 


14.5.7  Computer  Software  Components  (CSC)  Integration  and  Testing  Phase 

14.5.7.1  Purpose.  The  purpose  of  this  phase  is  to  integrate  units  of  code  that  have  been  entered 
in  the  development  configuration  and  to  perform  informal  tests  on  aggregates  of  integrated  units. 

14.5.7.2  Products.  The  following  products  are  normally  updated  and/or  produced  during  this  phase: 

a.  Software  test  procedures  report  (STPR) 

b.  Updated  computer  operator’s  manual  (CSOM) 

c.  Updated  software  user's  manual  (SUM) 

d.  Updated  computer  software  diagnostics  (CSDM) 

e.  Updated  software  programmer’s  manual  (SPM) 

f.  Updated  firmware  support  manual  (FSM) 

g.  Source  code 

h.  Object  code 

i.  Internal  review  reports 
j  Informal  integration  test  results. 

14.5.7.3  Test  Readiness  Review  (TRR)  and  Baseline.  The  purpose  of  the  TRR  is  to  review  the  informal 
test  results,  formal  test  procedures,  and  operations  and  support  documents  with  the  contracting  agencv 
and  :o  demonstrate  that 

a.  The  CSCI  test  procedure  is  complete. 


DI-MCCR-80016 

Dl-MCCR-80018 

Dl-MCCR-80019 

DI-MCCR-80020 

DI-MCCR-80021 

DI-MCCR-80022 


14-21 


b.  Each  CSCI  is  ready  for  formal  test. 

c.  The  CSOM,  SUM,  and  CSDM  adequately  address  the  operation  and  support  of  the  computer 
system . 

The  TRR  is  a  formal  review  and  will  be  officially  acknowledged  by  the  contracting  agency.  MIL- 
STD-1521A,  Appendix  F,  describes  the  process.  The  information  and  data  produced  during  this  phase 
become  part  of  the  development  configuration. 

14.5.7.4  Activities,  Plans,  and  Controls.  These  items  include  the  following: 

a.  Integrate  and  test  aggregates  of  units  in  a  top-down  sequence. 

b.  Compare  memory  and  processing  time  values  with  established  allocations. 

c.  Modify,  as  necessary,  all  controlled  or  baselined  documentation  based  on  memory,  processing 
time,  and  system  resources  comparisons. 

d.  Maintain  configuration  control  over  modified  documents. 

e.  Document  the  results  of  all  integration  testing  as  described  in  the  SSPM  or  SDP. 

f.  Update  design  documentation  and  code. 

g.  Complete  detailed  procedures  from  CSCI  testing. 

h.  Conduct  internal  inprocess  reviews. 

i  Update,  as  necessary,  the  CSDM,  SUM,  CSOM,  SPM,  and  FSM. 

14.5.7.5  Management  Documents.  No  necessary  management  documents  are  developed  during  this 
phase.  Existing  management  documents  should  be  reviewed  and  updated  as  required 

14.5.7.6  Quality  Factors.  Achieving  software  quality  requires  that  quality  be  built  in  during  the 
development  process  and  evaluated  during  each  phase.  The  appropriate  quality  factors  and  quality 
measurements  should  be  described  in  the  SQEP  and  the  quality  procedures.  Flow-down  of  quality  factors 
from  the  SRS  to  low-level  activities  is  a  requirement. 

14.5.7.7  Qualification/Evaluation  Requirements.  Formal  qualification  of  components  during  this 
phase  would  only  be  performed  on  those  units/CSCs  which  contain  functions  that  cannot  be  qualified 
at  the  CSCI  level.  Formal  tests  conducted  during  the  phase  require 

a.  Formal  test  procedures 

b.  Contracting  agency’s  approval  of  the  test  procedures 

c.  Test  performed  in  accordance  with  the  approved  test  procedures. 

The  contractor/developer  is  required  to  monitor  the  software  development  effort  for  consistency 
with  the  SDP,  SSPM,  SCMP,  and  SQEP  and  to  notify  the  contracting  agency  of  any  proposed  changes 
to  these  documents.  The  proposed  changes  will  be  subject  to  disapproval  by  the  contracting  agency. 
In  addition,  the  contractor/developer  shall  notify  the  contractor  agency  of  any  actions  t  r  procedures 
that  deviate  from  the  SDA,  SSPM,  SCMP,  or  SQEP. 

14.5.8  CSCI  Testing  Phase 

14.5.8.1  Purpose.  The  objective  of  this  phase  is  to  show  that  the  CSCI  satisfies  its  specific 
requirements,  e  g  .  functional,  interface,  performance,  and  quality. 
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14.5.8.2  Products.  During  this  phase,  the  contractor/developer  normally  produces  and/or  updates 
the  following  documents: 

a.  Completed  computer  software  operator’s  manual  (CSOM)  DI-MCCR-8001B 

b.  Completed  systems  user’s  manual  (SUM)  DI-MCCR-80019 

c.  Completed  computer  software  diagnostics  manual  (CSDM)  DI-MCCR-80020 

d.  Completed  software  programmer’s  manual  (SPM)  DI-MCCR-80021 

e.  Completed  firmware  support  manual  (FSM)  DI-MCCR-80022 

f.  Version  description  document  (VDD)  DI-MCCR-80013 

g.  Software  product  specification  (SPS)  DI-MCCR-80029 

h.  Software  test  reports 

i.  Record  of  internal  reviews 

j.  Updated  source  and  object  code 

14.5.8.3  Audits,  Reviews,  and  Baselines.  The  purpose  of  the  functional  configuration  audit  (FCA) 
is  to  demonstrate  to  the  contracting  agency  that  the  CSCI  was  successfully  tested  and  meets  the 
requirements  of  the  SRS  and  the  IRSs.  The  FCA  also  demonstrates  to  the  contracting  agency  that  the 
CSOM,  SUM,  and  CSOM  adequately  address  the  operation  and  support  of  the  computer  system.  The 
contractor  will  present  the  CSOM,  SUM,  and  CSDM  at  the  FDA. 

The  purpose  of  the  physical  configuration  audit  (PCA)  is  to  demonstrate  to  the  contracting  agency 
that  the  SPS  is  complete  and  reflects  an  up-to-date  technical  description  of  the  CSCI.  The  contractor 
shall  present  the  SPS,  VDD,  and  source  code  at  the  PCA. 

The  configuration  identification  documents  for  HWCIs  and  CSCIs  comprise  a  system  from  a  single 
product  baseline.  When  the  FCA  and  PCA  for  each  CSCI  have  been  completed  and  authenticated 
by  the  contracting  agency,  the  SPS  for  the  CSCI  will  be  entered  into  the  product  baseline.  See  MIL- 
STD-1521B  for  information  concerning  the  FCA  and  PCA  process. 

14.5.8.4  Activities,  Plans,  and  Controls.  The  contractor/developer/IV&V  contractor  shall  test  the 
CSCI  using  formal  test  procedures  approved  by  the  contracting  agency. 

Individuals  sufficiently  independent  from  the  software  developer  shall  perform  formal  tests  on 
each  CSCI  in  accordance  with  the 

a.  Formal  test  plans 

b.  Formal  test  cases 

c.  Formal  test  procedures. 

The  test  reports  accumulated  by  the  independent  test  group  shall  report  the  results  of  all  formal 
tests.  The  test  reports  shall  include 

a.  Summary  of  tests  results 

b.  Detail  of  test  results 

c.  Evaluation  of  test  results 

d  Recommendations 

e.  Test  procedure  deviations. 
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The  contractor/developer  shall 

a.  Make  necessary  revisions  to  the  design  documentation.  r' ,v‘\ 

•  m 
C' 

b.  Make  necessary  revisions  to  the  code. 

c.  Perform  all  necessary  retests. 

d.  Update  all  SDFs. 

The  contractor/developer  shall  identify  the  exact  version  of  each  deliverable  CSC1  and  the  interim 
changes  occurring  between  versions.  The  identification  shall  include 

a.  Inventory  of  materials 

b.  Inventory  of  CSCI  contents 

c.  Class  1  changes  installed 

d.  Class  2  changes  installed 

e.  Adaptation  data 

f.  Operational  description 

g.  Installation  instructions 

h.  Possible  problems  and  known  errors. 

The  contractor/developer  shall  conduct  internal  in-process  reviews  during  this  phase  and  make 
all  changes  based  on  the  results  of  the  internal  review  prior  to  presenting  the  formal  test  results  and 
completed  operation  and  support  documents  to  the  contracting  agency. 

14.5.8.5  Management  Documents.  No  new  management  documents  are  developed  during  this  phase.  0b* 

14.5.8.6  Quality  Factors.  The  quality  factors  specified  in  the  software  requirements  specification 
(SRS)  apply  during  the  CSCI  testing  phase.  The  applicable  quality  factors  and  the  quality  measures 
required  to  support  the  realization  of  the  quality  factors  should  be  specified  in  the  SQEP  and  the  quality 
procedures. 

14.5.8.7  Qualification  Requirements.  The  contractor/developer  shall  conduct  formal  tests  on  each 
CSCI  to  show  that  the  CSCI  satisfies  its  specified  requirements.  Personnel  conducting  CSCI  tests  and 
analyzing  formal  test  data  shall  be  sufficiently  independent  from  the  individuals  responsible  for 
development  to  permit  objective  testing. 


14.6  SOFTWARE  QUALITY  EVALUATION 
14.6.1  Purpose 

The  purpose  of  the  software  quality  evaluation  plan  (SQEP)  is  to  describe  the  organization  and 
procedures  to  be  used  by  the  contractor  to  determine  the  quality  of  the  software  and  associated 
documentation. 

The  SQEP  is  used  by  the  government  to  monitor  the  procedures,  management,  and  work  effort 
of  the  contractors'  organizations  performing  software  quality  evaluation.  The  SQEP  and  the  quality 
procedures  are  subject  to  disapproval  by  the  contracting  agency. 
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14.6.1.1  Quality  Evaluation.  The  contractor  developer  shall  perform  the  planning  and  implement 
internal  procedures  to 

a.  Evaluate  the  requirements 

b  Evaluate  the  methodology 

c.  Evaluate  the  products 

d.  Provide  feedback  and  recommendation 

e.  Detect,  report,  and  track  problems. 

The  method  for  accomplishing  the  above  shall  be  specified  in  the  SQEP, 


14.6.2  Internal  Reviews 

14.6.2.1  Internal  Reviews.  The  contractor  shall  conduct  internal  reviews  to  determine  the  following: 

a.  Conformance  to  the  methodologies  proposed  in  the  contractor’s  developer’s  planning  document 

b.  Compliance  with  the  methodologies  proposed  in  DoD-STD-2167 

c.  Adequacy  of  the  contractor’s  process  and  methodologies  to  produce  quality  products  that  will 
meet  established  requirements 

d.  Compliance  of  process  with  methodologies 

e.  Adequacy  of  in-process  reviews  to  evaluate  products. 

14.6.2.2  Evaluation  Criteria.  The  contractor/developer  shall  use  the  following  evaluation  criteria: 

a.  Adherence  to  required  format 

b.  Compliance  with  contracted  requirements 

c.  Internal  consistency 

d.  Understandability 

e.  Technical  adequacy. 

14.6.2.3  Internal  Reviews — All  Phases.  The  contractor/dev  eloper  shall  conduct  the  following  reviews 
during  all  phases  of  the  software  development  cycle. 

a.  Review  newly  prepared  or  revised  SDP,  SSPM,  SCMP,  and  SQEP  for 

1.  Adherence  to  required  format 

2.  Compliance  with  contracted  requirements 

3.  Internal  consistency 

4.  Understandability 

5.  Technical  adequacy 

6  Degree  of  completeness. 

b.  Review  the  activities  and  the  tools,  procedures,  and  methodologies  employed  during  the  phase 
for  consistency  with  the  contractor’s  soltware  development  plans,  included  in  tins  review  shall 
be  evaluation  of 
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1.  Software  configuration  management 

2.  Software  development  library 

3.  Documentation  contract 

4.  Storage  and  handling  of  media 

5.  Control  of  nondeliverables 

6.  Risk  management 

7.  Corrective  action 

8.  Conformance  to  standards  and  procedures. 

14.6.3  Internal  Review — Software  Requirements  Analysis 

The  contractor/developer  shall  conduct  internal  reviews  during  the  software  requirements  phase. 

14.6.3.1  The  OCD  shall  be  reviewed  for 

a.  Adherence  to  required  format 

b.  Compliance  with  contractual  requirements 

c.  Internal  consistency 

d.  Understandability 

e.  Technical  adequacy 

f.  Degree  of  completeness 

g.  Consistency  with  the  SSS 

h.  High-level  understanding. 

14.6.3.2  The  ongoing  SRS  and  IRSs  shall  be  reviewed  for 

a.  Adherence  to  required  format 

b.  Compliance  with  contractual  requirements 

c.  Internal  consistency 

d.  Understandability 

e.  Technical  adequacy 

f.  Degree  of  completeness 

g.  Traceability  of  requirements  to  system  specification 

h.  Consistency  of  interface  requirements  with  specifications  for  interfacing  elements 

i.  Consistency  of  SRS  and  IRSs  with  one  another 

j.  Testability  of  functional,  performance,  and  intertace  requirements 

14.6.3.3  The  following  management  documents  shall  be  reviewed  for  adequate  control,  technical 
feedback,  and  management  feedback. 
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a.  Software  configuration  management 

b.  Software  development  library 

c.  Documentation  control 

d.  Storage  and  handling  of  media 

e.  Control  of  nondeliverables 

f.  Risk  management 

g.  Corrective  action 

h.  Conformance  to  all  approved  standards  and  procedures. 

14.6.4  Internal  Review — Preliminary  Design 

14.6.4.1  Process.  The  contractor/developer  shall  conduct  internal  reviews  during  the  preliminary 
design  phase.  The  process  shall  be  reviewed  for  adequate  performance  in  the  following  areas: 

a.  Software  configuration  management 

b.  Software  development  library 

c.  Documentation  control 

d.  Storage  and  handling  of  media 

e.  Control  of  nondeliverables 

f.  Risk  management 

g.  Corrective  action 

h.  Conformance  to  all  approved  standards  and  procedures. 

14.6.4.2  General  Product  Reviews.  All  products  shall  be  reviewed  for 

a.  Adherence  to  required  format 

b.  Compliance  with  contractual  requirements 

c.  Internal  consistency 

d.  Understandability 

e.  Technical  adequacy 

f.  Degree  of  completeness. 

14.6.4.3  Specific  Product  Reviews.  The  specific  products  shall  be  reviewed  for  the  following 
characteristics. 

a.  The  top-level  design  and  STLDD  shall  be  reviewed  for  traceability  to  the  SRS  and  IDRs. 

b.  The  STP  shall  be  reviewed  for 

1.  Adequate  test  coverage 

2.  Consistency  with  the  SDP 

3.  Adequate  planning. 


c.  The  preliminary  versions  of  the  CSOM,  SUM,  and  CSDM  will  be  reviewed  for 

1.  Consistency  with  SRS 

2.  Appropriate  content 

3.  Consistency  with  one  another. 

d.  The  preliminary  CRISD  shall  be  reviewed  for 

1.  Consistency  with  government  support  concepts 

2.  Adequacy  of  support  planning. 

14.6.5  Internal  Review — Detailed  Design 

14.6.5.1  Process.  The  contractor/developer  shall  conduct  internal  reviews  during  the  detailed  design. 
The  process  shall  be  reviewed  for 

a.  Software  configuration  management 

b.  Software  development  library 

c.  Documentation  control 

d.  Storage  and  handling  of  media 

e.  Control  of  nondeliverables 

f.  Risk  management 

g.  Corrective  action 

h.  Conformance  to  all  approved  standards  and  procedures. 

14.6.5.2  General  Product  Reviews.  All  products  shall  be  reviewed  for 

a.  Adherence  to  required  format 

b.  Compliance  with  contractual  requirements 

c.  Internal  consistency 

d.  Understandability 

e.  Technical  adequacy 

f.  Degree  of  completeness. 

14.6.5.3  Specific  Product  Reviews.  The  specific  products  shall  be  reviewed  for  the  following 
characteristics. 

a.  Review  of  evolving  detailed  design  and  the  SDDD,  IDDs,  and  DBDDs,  as  applicable  for 

1.  Traceability  to  SRS,  IRS,  and  STLDD 

2.  Use  of  appropriate  design  techniques 

3.  Consistency  with  one  another. 

b.  Review  one  STP  for 


1.  Adequate  test  coverage 


2.  Consistency  with  design. 

c.  Review  software  development  files  for  accuracy  of  schedule  and  status. 

d.  Review  unit  test  cases  for 

1.  Traceability  to  the  STP 

2.  Adequate  test  coverage 

3.  Consistency  with  design. 

e.  Review  integration  test  cases  for 

1.  Traceability  to  the  STP 

2.  Adequate  test  coverage 

3.  Consistency  with  design  documentation. 

f.  Review  the  updated  CSOM,  SUM,  and  CSDM  for 

1.  Consistency  with  requirements  and  design 

2.  Appropriateness  of  content 

3.  Consistency  with  one  another. 

g.  Review  the  completed  CRISD  for 

1 .  Consistency  with  government  support  concepts 

2.  Adequacy  of  support  planning. 

h.  Review  the  SPM  and  FSM  for 

1.  Consistency  with  design  documentation 

2.  Appropriateness  of  content  for  support  personnel. 


14.6.6  Internal  Review — Code  and  Unit  Testing 

14.6.6.1  Process.  The  contractor/developer  shall  conduct  internal  reviews  during  the  code  and  unit 
testing  phase.  The  process  shall  be  reviewed  for 

a.  Software  configuration  management 

b.  Software  development  library 

c.  Documentation  control 

d.  Storage  and  handling  of  media 

e.  Control  of  nondeliverables 
t  Risk  management 

g.  Corrective  action 

h.  Conformance  to  all  approved  standards  and  procedures. 

14.6.6.2  («eneral  Product  Reviews.  All  products  shall  be  reviewed  for 
a.  Adherence  to  required  format 
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b.  Compliance  with  contractual  requirements 

c.  Internal  consistency 

d.  Understand  ability 

e.  Technical  adequacy 

f.  Degree  of  completeness. 


14.6.6.3  Specific  Product  Reviews.  The  specific  products  shall  be  reviewed  for  the  following 
characteristics: 

a.  The  evolving  and  completed  source  code  shall  be  reviewed  for 

1.  Compliance  with  coding  standards 

2.  Traceability  to  detailed  design. 

b.  The  software  development  files  shall  be  reviewed  for 

1.  Accuracy  of  status  and  schedule 

2.  Unit  test  procedures 

3.  Unit  test  results 

4.  Traceability  to  unit  test  plans 

5.  Traceability  to  unit  test  cases 

6.  Readiness  for  units  to  be  placed  under  CM. 

c.  The  STLDD,  SDDD,  IDDs,  and  DBDDs,  as  applicable,  shall  be  reviewed  for 

1.  Traceability  to  SRS 

2.  Use  of  appropriate  design  techniques 

3.  Consistency  with  one  another. 

d.  The  updated  source  code,  as  applicable,  shall  be  reviewed  for 

1.  Compliance  with  coding  standards 

2.  Consistency  with  the  updated  detailed  design  documentation. 

e.  The  informal  integration  test  procedures  shall  be  reviewed  for 

1.  Traceability  to  CSC  integration 

2.  Adequate  test  coverage 

3.  Consistency  with  design  documents. 

f.  The  preliminary  design  STPR  shall  be  reviewed  for 

1.  Traceability  to  the  STP  and  STD 

2.  Adequate  test  coverage 

3.  Consistency  with  design  documents. 

g.  The  updated  CSOM,  SUM,  and  CSDM  shall  be  reviewed  for 
1 .  Consistency  with  requirements  and  design  documents 
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2.  Appropriateness  of  content 

3.  Consistency  with  one  another. 

h.  The  updated  SPM  and  FSM  shall  be  reviewed,  as  applicable,  for 

1.  Consistency  with  design  documentation 

2.  Appropriateness  of  content  for  support  personnel. 

14.6.7  Internal  Review-CSC  Integration  and  Testing 

14.6.7.1  Process.  The  contractor/developer  shall  conduct  internal  reviews  during  the  CSC  integration 
testing  phase.  The  process  will  be  reviewed  for 

a.  Software  configuration  management 

b.  Software  development  library 

c.  Documentation  control 

d.  Storage  and  handling  of  media 

e.  Control  of  nondeliverables 

f.  Risk  management 

g.  Corrective  action 

h.  Conformance  to  all  approved  standards  and  procedures. 

14.6.7.2  General  Product  Review.  All  products  shall  be  reviewed  for 

a.  Adherence  to  required  format 

b.  Compliance  with  contractual  requirements 

c.  Internal  consistency 

d.  Understandability 

e.  Technical  adequacy 

f.  Degree  of  completeness. 

14.6.7.3  Specific  Product  Reviews.  The  specific  product  shall  be  reviewed  for  the  following 
characteristics: 

a.  The  informal  test  results  of  CSC  integration  will  be  reviewed  for 

1.  Traceability  to  CSC  test  cases 

2.  Traceability  to  CSC  test  procedures 

3.  Correct  performance  of  the  integrated  CSCI 

4.  Readiness  for  the  CSCI  to  undergo  formal  testing. 

b.  The  updated  STLDD,  SDDD,  IDDs,  and  DBDDs  shall  be  reviewed  for 

1.  Traceability  to  software  requirements 

2.  Use  ot  appropriate  design  techniques 


3.  Consistency  with  one  another. 

c.  The  source  code  shall  be  reviewed  for 

1.  Compliance  with  coding  standards 

2.  Consistency  with  update  design  documentation. 

d.  The  updated  software  development  files  shall  be  reviewed  for  accuracy  of  status  and  schedule. 

e.  The  completed  STPR  shall  be  reviewed  for 

1.  Traceability  to  the  STP  and  STD 

2.  Adequate  test  coverage 

3.  Consistency  with  design  documentation. 

f.  The  updated  CSOM,  SUM,  and  CSDR  shall  be  reviewed  for 

1 .  Consistency  with  requirements  and  design  documentation 

2.  Appropriateness  of  content 

3.  Consistency  with  one  another. 

g.  The  updated  SPM  and  FSM  shall  be  reviewed  for 

1.  Consistency  with  design  documentation 

2.  Appropriateness  of  content  for  support  personnel. 

14.6.8  Internal  Review — CSCI  Testing 

14.6.8.1  Process.  The  contractor/developer  shall  conduct  internal  reviews  during  the  CSCO  testing 
phase.  The  process  will  be  reviewed  for 

a.  Software  configuration  management 

b.  Software  development  library 

c.  Documentation  control 

d.  Storage  and  handling  of  media 

e.  Control  of  nondeliverables 

f.  Risk  management 

g.  Corrective  action 

h.  Conformance  to  all  approved  standards  and  procedures. 

14.6.8.2  General  Product  Review.  All  products  shall  be  reviewed  for 

a.  Adherence  to  required  format 

b.  Compliance  with  contractual  requirements 

c.  Internal  consistency 

d.  Understandability 

e.  Technical  adequacy 

f.  Degree  of  completeness. 
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14.6.8.3  Specific  Product  Review.  The  specific  products/activities  shall  be  reviewed  for  Hie  following 
characteristics. 

a.  The  CSCI  testing  shall  be  monitored  to  ensure  that 

1.  It  is  performed  using  the  current  controlled  version  of  the  code 

2.  It  is  conducted  in  accordance  with  approved  test  plans,  descriptions,  and  procedures 

3.  Necessary  retesting  is  accomplished. 

b.  The  STRs  shall  be  reviewed  for  traceability  of  the  CSCI  test  results  to  the  CSCI  test  plans, 
test  cases,  and  test  procedures. 

c.  The  updated  STLDD,  SDDD,  IDDs  and  DBDDs,  as  applicable,  shall  be  reviewed  for 

1.  Traceability  to  software  requirements 

2.  Use  of  appropriate  design  techniques 

3.  Consistency  with  one  another. 

d.  The  updated  source  code,  as  applicable,  shall  be  reviewed  for 

1.  Compliance  with  coding  standards 

2.  Consistency  with  the  updated  detailed  design  documentation. 

e.  The  software  development  files  shall  be  reviewed  for  accuracy  of  status  and  schedule. 

f.  The  SPS  shall  be  reviewed  for  incorporation  of  design  documentation  and  software  listings 
consistent  with  the  “as-built”  software. 

g.  The  VDD  shall  be  reviewed  for  accuracy  in  reflecting  the  exact  version  of  each  CSCI. 

h.  The  completed  CSOM,  SUM,  and  CSDM  shall  be  reviewed  for 

1.  Consistency  with  the  SPS 

2.  Appropriateness  of  content 

3.  Consistency  with  one  another. 

i.  The  SPM  and  FSM  shall  be  reviewed  for 

1.  Consistency  with  design  documentation 

2.  Appropriateness  of  content. 

14.6.9  Evaluation 

14.6.9.1  Formal  Reviews  and  Audits.  The  contractor/developer  shall  evaluate  the  planning  and 
preparation  performed  for  each  formal  review  and  audit  to  ensure  that 

a.  Required  products  (e.g.  the  data  package)  will  be  available  for  review 

b.  Necessary  resources  and  material  are  available 
c  Meeting  agenda  is  coordinated 

d  Cochairpersons  are  designated 

e.  Action  items  and  action  items  sources  are  recorded. 


The  following  formal  design  reviews  and  audits  are  normally  conducted  during  the  system  life  cycle: 

a.  System  requirement  reviews  (SRR) 

b.  System  design  review  (SDR) 

c.  Software  specification  review  (SSR) 

d.  Preliminary  design  review  (PDR) 

e.  Critical  design  review  (CDR) 

f.  Test  readiness  review  (TRR) 

g.  Functional  configuration  audit  (FCA) 

h.  Physical  configuration  audit  (PCA) 

i.  Formal  qualification  review  (FQA) 

j.  Production  readiness  review  (PRR). 

14.6.9.2  Evaluation  of  Subcontractor  Products.  Prior  to  accepting  software  or  documentation 
developed  from  a  subcontractor,  the  prime  contractor  shall  evaluate  the  products  for 

a.  Completeness 

b.  Technical  adequacy 

c.  Compliance  with  subcontract  requirements. 

14.6.9.3  Quality  Records.  The  contractor/developer  shall  prepare  and  maintain  records  of  each  quality 
evaluation  performed;  quality  records  shall  identify 

a.  Date  of  evaluation 

b.  Evaluation  participants 

c.  Items  or  activities  reviewed 

d.  Objectives  of  the  evaluation 

e.  Detected  problems 

f.  Recommendations. 

14.6.9.4  Quality  Reporting.  The  contractor/ developer  shall  prepare  reports  that  provide  to  contractor 
management  the  results  and  recommendations  from  the  quality  evaluations.  The  quality  evaluation 
reports  shall  identify 

a.  Evaluation  activity 

b.  Detected  problems 

c.  Remedial  action 

d.  Trends 

e.  Recommended  changes. 

14.6.9.5  Corrective  Action  System.  The  contractor/developer  shall  implement  a  corrective  action 
system  for  all  software  and  documentation  that  has  been  placed  under  contractor  or  government  control. 
The  corrective  action  system  shall  include  provisions  for 
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a.  Reporting  problems 

b.  Analyzing  problems 

c.  Classifying  problems  by  category  and  priority 

d.  Identifying  corrective  action 

e.  Identifying  trends 

f.  Analyzing  trends 

g.  Authorizing  corrective  action 

h.  Implementing  corrective  action 

i.  Reevaluating  the  problem  after  corrective  action 

j.  Tracking  problems 

k.  Closing  out  problems 

l.  Providing  government  visibility  into  critical  problems  based  on  categorization,  priority  schemes, 
and  problems/change  reports. 

14.6.9.6  Quality  Cost  Data.  The  contractor/developer  shall  collect  and  analyze  the  document  data 
relative  to  the  cost  of  detecting  and  correcting  errors  in  all  software  and  documentation  that  have  been 
placed  under  contractor  or  government  control.  The  specific  data  to  be  collected  and  the  analyses  to 
be  performed  shall  be  proposed  by  the  contractor  in  either  the  SQEP  or  the  SDP  and  shall  be  subject 
to  contracting  agency  approval. 

14.6.9.7  Products — Software  Quality  Evaluation.  The  following  are  included  in  the  quality  evaluation 
considerations. 

a.  Quality  records.  The  contractor/developer  shall  prepare  and  maintain  records  of  each  quality 
evaluation. 

b.  Quality  reports.  The  contractor  shall  prepare  and  maintain  reports  that  summarize  the  results 
and  recommendations  of  quality  evaluations  performed.  These  reports  shall  be  made  available 
for  government  review. 

c.  Certification.  The  contractor/developer  shall  collect  and  make  available  for  government 
inspection  evidence  indicating  the  compliance  with  the  requirements  of  the  contract. 

d.  Independence.  Each  software  quality  evaluation  shall  be  performed  by  individuals  who  have 
sufficient  responsibility,  authority,  resources,  and  independence  to  accomplish  objective 
evaluation  of  the  products  and  activities  being  evaluated.  The  degree  of  evaluation  independence 
shall  be  specified  in  the  SQEP  or  SDP. 

14.6.10  Software  Project  Planning  and  Control 

14.6.10.1  Sizing  and  Timing.  The  contractor/developer  shall  derive  sizing  and  timing  parameters 
for  each  CSC1  including  minimum  reserve  capacities.  The  contractor  will  monitor  these  parameters 
and  reallocate  as  necessary  to  meet  requirements  specified  in  the  SRS. 

14.6. 10.2  Status  and  Cost  Report.  The  contractor/developei  shall  maintain  cost  and  schedule  forecast . 
analysis,  and  reports.  These  reports  shall  conform  to  the  WHS. 
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14.6.10.3  Test  Documentation  Control.  The  contractor/developer  shall  establish  internal  contr'l 
over  approved  STP,  STD,  and  STPRs.  The  contracting  agency  shall  be  notified  of  any  proposed  changes 
to  these  documents,  and  the  contractor  shall  obtain  approval  before  making  any  changes. 


14.6.10.4  Software  Development  Library.  The  contractor/developer  shall  establish,  implement,  and 
control  the  content  of  the  software  development  library. 

14.6.10.5  Risk  Management.  The  contractor/developer  shall  establish  and  implement  the  risk 
management  procedures  specified  in  the  SDP  for  controlling  risk.  The  procedures  shall  include 

a.  Identifying  the  risk  areas  and  the  consistent  risk  factors  in  each  area 

b.  Assessing  the  probability  of  occurrence  and  the  potential  damage  associated  with  each  risk  factor 

c.  Assigning  appropriate  resources  to  reduce  the  risk  factors 

d.  Identifying  and  analyzing  the  alternatives  available  for  reducing  the  risk  factors 

e.  Selecting  the  most  promising  alternative  for  each  risk  factor 

f.  Planning  implementations  of  the  selected  alternatives  for  each  risk  factor 

g.  Obtaining  feedback  to  determine  the  success  of  the  risk  reducing  action  for  each  risk  factor. 
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SECTION  15 
CONTRACTING 
D.  Lumpkins,  Code  217 


15.1  INTRODUCTION 


15.1.1  References  (listed  by  topics) 

Acquisition  Planning  and  APs 


ADPE  Acquisition 


Competition  Requirements 


Contracted  Advisory  and  Assistance  Services 
Contracting  Officer’s  Technical  Representative 

Data 


Incremental  Funding 
Major  Systems  Acquisition 


FAR  Part  7 
DFARS  Part  7 
NARSUP  Part  7 
NOSCNOTE  4200 
PM  19-86/Code  217 

SECNAVINST  5231. IB 
SECNAVINST  5236. IB 
SECNAVINST  5236.2A 
SECNAVINST  5237.1 
SECNAVINST  5238. IB 
NAVSUPINST  4284.3 
NOSCINST  5230. 1C 

FAR  Part  6 
DFARS  Part  6 
NARSUP  Part  6 
FAR  15.5 

NOSCINST  4200. 6A 
NOSINST  3900. 1C 
NOSINST  4200.6S 
PM  13-85/Code  217 
C1CA 

SECNAVINST  4200.31  A 

NOSCINST  4330.1 
NAVSUPINST  4330. 6B 
NOSCINST  4235.1 

DFARS  53.303-70 
NAVMATINST 
4000. 15A 
DoDINST  5010.12 
NOSCISNT  4000.11) 

NOSCINST  7300. 3A 

DoDINST  5000.1(D) 
DoDINST  5000.2 


Multiple  Funding  &  Funding  Plan 
Nonpersonal  Services  Determination 
Patent  Rights 


NOSCINST  7300.7 
NOSCINST  4200.4B 


FAR  27.3 

NOSCINST  4430.4C 


Proprietary  Material 

Security  Classification  Specification 

NOSCINST  3900.6b 

FAR  53.204-l(a) 

NOSC  TD  490A 

Service  Contracts 

NAVSUPINST  4300.6B 
NOSCINST  4330.1 

Signature  Authority  &  Priority  Designator 

Small  Business 

Small  Purchase  Procedures 

NOSCINST  4614. IB 

FAR  19.6 

FAR  Part  10 

FAR  Part  13 

DFARS  Part  13 
NARSUP  Pan  13 
SUPARS  P-560 
OPNAVINST  4614. IF 
PM  13-86/Code  217 
PM  14-86/Code  217 
PM  15-86/Code  217 

Source  Selection  Planning/Tech  Eval 

Specifications 

FAR  Pan  15 

FAR  part  10 
MIL-STD-490 

Statement  of  Work 

MIL-HDBK-245B 

M1L-S-83490 

MIL-STD-490 

DoD-D-1000 

Stub  Requisition 

FAR  53.301-4 

FAR  13.5 

PM/Code  217 

Supply  Department  Organization 

Unauthorized  Commitment  of  Appropriated  Funds 
Unsolicited  Contractor  Proposals 

NOSCNOTE  5400 

NOSCINST  4235.1 

NOSCINST  3900. 1C 

NOTE:  PM  stands  for  procurement  memoranda;  available  from  Code  217  (Policy  and  fanning 
Branch),  x7141.  Instructions  and  Notices  are  available  from  Alma  Savage.  Code  132,  x65_3. 

I 

15.1.2  Outline 

Introduction 

References 

I  Outline 

Summary 
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Glossary  of  Acquisition  Terms 
Contracting  Conditions  and  Environment 
Legislation  Affecting  Acquisition 
Deputy  for  Small  Business 
Data  Management  Office 

Contracting  Officer’s  Technical  Representative’s  Duties  and  Responsibilities 
Unauthorized  Commitments 
Advance  Acquisition  Planning  and  APs 
Background 

Benefits  of  Contract  Planning 
Timing  Considerations 
Acquisition  Plans 
Further  Presolicitation  Preparation 
Statement  of  Work 
Specifications 
Technical  Evaluation  Plan 
Other  Than  Full  and  Open  Competition 

Sole  Source  Justification  and  Justification  and  Authorization 
Brand  Name  or  Equal 
Contract  Types 
References 
Service  Contracts 
Indefinite  Delivery  Contracts 
CPFF  Contracts  (Completion  versus  Term) 

Solicitation  Phase 
Administration  Phase 
Special  Considerations 
ADP  Acquisitions 
Small  Purchase  Procedures 
Sole  Source 
Purchase  Descriptions 
Buy  American  Act 
Suggested  Sources 
Technical  Evaluation 
Offsite  Repairs 
Onsite  Repairs 
Stubs  Status 
Patents 

Unsolicited  Contractor  Proposals 

Appendix  ISA  Contents  of  Acquisition  Requirements  Package  (ARP) 

l SB  The  NOSC  Stub  Requisition  Form  and  Instructions  for  Its  Use 
15C  Internal  Approval  Control  Points  for  Commodities/Documentation 
at  NOSC 

15.1.3  Summary 

This  section  is  devoted  to  the  topic  of  contracting.  It  is  inn  tided  to  be  an  abstract  of  a  subject 
so  massive  and  so  fluid  that  a  full  discussion  of  it  would  be  impractical.  We  approached  the  undue 
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of  this  section  aware  that  the  contracting  process  cannot  be  fully  defined  because  there  is  constant 
revision  and  change.  Therefore,  we  seek  to  provide  the  program  manager  with  the  fundamental  concepts 
of  the  acquisition  process  (the  language,  the  references,  and  the  names  of  knowledgeable  individuals 
in  each  subject  area)  rather  than  a  detailed  discussion  on  each  topic.  The  references  cited  herein  will 
provide  program  management  with  the  information  required  to  gain  a  more  complete  understanding 
of  individual  aspects  of  the  acquisition  cycle. 

NOSCINST  4200.5  “Procurement  Requirements  Package  Handbook”  (30  August  1978)  has  been 
cancelled  and  is  being  rewritten.  Its  successor,  the  Acquisition  Requirements  Package  Handbook ,  will 
become  the  comprehensive  local  instruction  on  the  subject  of  contracting,  providing  detailed  instructions 
for  preparing  the  acquisition  requirements  package. 

Be  advised  that  many  important  changes  in  DoD  acquisition  have  taken  place.  Do  not  rely  on 
any  information  currently  in  print  or  on  previous  experience  to  guide  you.  Until  the  new  handbook 
is  issued,  contact  your  Contracts  Branch  for  specific  guidance.  Do  not  rely  on  any  information  currently 
in  print  or  on  previous  experience  to  guide  you.  Figures  15.1  and  15.2  show  the  organization  charts 
for  NOSC’s  Supply  Department  and  Contracts  Division,  respectively. 

This  section  is  ordered  in  a  roughly  chronological  fashion.  First  come  the  glossary  (15.2),  which 
introduces  you  to  the  language  of  contracting,  and  subsection  15.3,  which  sketches  the  environment, 
conditions,  and  responsibilities  involved  in  the  contracting  process.  After  acquisition  planning  documents 
are  reviewed  (15.4),  the  specific  presolicitation  preparations  that  include  the  statement  of  work, 
specifications,  and  the  technical  evaluation  plan  are  presented  (15.5).  Approaches  other  than  full  and 
open  competition  are  considered  in  subsection  15.6;  these  include  sole  source  justification  and  the 
brandname  —  or  —  equal  requirement.  Subsection  15.7  reviews  the  types  of  contracts.  Then  the 
solicitation  and  administration  phases  are  briefly  discussed  (15.8  and  15.9).  The  section  concludes  with 
special  contracting  considerations  (15.10)  and  the  appendixes. 


15.2  GLOSSARY  OF  ACQUISITION  TERMS 


Acquisition 

Administrative  Change 

Administrative 
Contracting  Officer  (ACO) 

ADP/ADPE 

Allocate 


AP 

ARP 

AR/DRRB 


Buying,  leasing,  renting,  or  otherwise  obtaining  supplies  or  services 
to  meet  needs  of  the  government. 

Contract  modification  signed  only  by  the  Contracting  Officer;  it 
has  no  effect  on  price,  performance,  or  delivery. 

A  government  contracting  officer,  often  at  an  installation  other 
than  the  one  which  made  the  contract,  who  handles  the  business 
administration  of  the  contract.  For  the  larger  primes,  the  ACO 
is  commonly  resident  at  the  prime’s  facility. 

Automatic  data  processing/equipment 

To  assign  an  item  of  cost,  or  a  group  of  items  of  cost,  to  one  or 
more  cost  objectives.  This  term  includes  both  direct  assignment 
of  cost  and  the  reassignment  of  a  share  from  an  indirect  cost  pool. 

Acquisition  plan 

Acquisition  requirements  package  (aka:  procurement  requirements 
package  (PRP)). 

Acquisition  requirements/Data  Requirements  Review  Board  (same 
as  DRRB) 


15-6 


ASN(S&L) 

Bid 

Bidders  (Mailing)  List 
(Master  Bidders  List) 


Assistant  Secretary  of  the  Navy  (Shipbuilding  &  Logistics) 

A  prospective  contractor’s  (bidder’s)  offer  in  reply  to  a  sealed  bid 
solicitation  document  “Invitation  for  Bid.”  Needs  only 
government  acceptance  to  constitute  a  binding  contract. 

List  of  sources  maintained  by  the  contracting  office  from  which 
bids  or  proposals  or  quotations  can  be  solicited. 


Blanket  Purchase  Agreement  A  negotiated  contractual  agreement  issued  by  Code  21 1  between 
(BP A)  a  contractor  and  the  government  under  which  individual  calls  not 

exceeding  $10,000,  except  for  subsistence,  may  be  placed  during 
a  specified  period  of  time  and  within  a  stipulated  aggregate 
amount,  if  any.  This  simplified  purchase  method  is  designed  to 
fill  anticipated  repetitive  needs  for  small  quantities  of  supplies  or 
services,  which  are  generally  ordered  by  placing  calls  orally,  thereby 
eliminating  the  need  for  issuing  individual  purchase  documents; 
as  a  result,  administrative  costs  are  reduced. 


Buy  American  Act  Federal  statute  imposing  restrictions  on  placing  contracts  with 

manufacturers  who  would  deliver  items  not  substantially  produced 
in  the  United  States. 


CAS 

CDRL 

CFE 

Change  Order 


Commerce  Business  Daily 
(CBD) 


Competition  Advocate 

Competition  Advocate 

Review  Board  (CARB) 


Cost  accounting  standards 

Contract  data  requirements  list  —  DD  form  1423 

Contractor-furnished  equipment 

Contract  modification  signed  only  by  the  contracting  officer 
directing  the  contractor  to  take  certain  action  within  the  scope 
of  the  contract.  Usually  affects  price,  performance,  and/or 
delivery. 

Daily  publication  by  Department  of  Commerce  synopsizing  all 
(except  classified  purchases  and  certain  other  specifically  excepted 
actions)  government  solicitations  in  excess  of  $25,000  and  awards 
exceeding  $100,000. 

At  NOSC,  Commanding  Officer. 

Committee  assembled  to  review  non-competitive  requirements  over 

$100,000. 


Contract 


Contract  Modification 


Contract  Type: 


Simply  an  agreement  between  the  government  and  contractor 
expressing  terms  and  conditions  affecting  price,  performance,  and 
delivery. 

Written  action  on  standard  form  30  changing  some  part  of  a 
contract.  A  formal  revision  of  the  terms  of  a  contract,  either  within 
or  outside  the  scope  of  the  agreement.  Includes  change  orders. 
See  also  Supplemental  Agreement. 

Refers  to  specific  pricing  arrangements  between  the  go\ernment 
(buyer)  and  the  performing  contractor(s)  (sellers)  for  payment  for 
work  performed  under  contracts.  The  compensation  arrangements 
include  firm  lixed-price,  fixed-price  incentive,  cost-plus- fixed- fee, 
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Basic  Ordering 
Agreement  (BOA) 


b.  Cost-Reimbursement 
Contracts 


c.  Fixed-Price  Contracts 


d.  Indefinite  Delivery 
Type  Contracts 
(IDTC) 


e.  Letter  Contract 


f.  Special  Purpose 


g.  Term  or 
Completion 


cost-plus-incentive-fee,  and  several  others.  Among  special 
arrangements  that  use  fixed-price  or  cost-reimbursement  pricing 
provisions  are  contract  types  called  indefinite  delivery  contracts, 
basic  ordering  agreements,  letter  contracts,  and  others. 

A  written  instrument  of  understanding  (not  a  legally  enforceable 
contract  per  se)  between  a  contractor(s)  and  the  government.  Sets 
forth  the  contract  clauses  applicable  to  future  procurements 
entered  into  between  the  parties  during  the  term  of  the  basic 
agreement.  Used  to  eliminate  extensive  and  costly  negotiation  when 
a  substantial  number  of  separate-contracts  may  be  entered  into 
with  a  contractor  over  a  definite  period  of  time. 

In  general,  category  of  contracts  whose  use  is  based  on  payment 
by  the  government  to  a  contractor  of  allowable  costs  as  prescribed 
by  the  contract.  Normally  only  “best  efforts”  of  the  contractors 
are  involved.  This  category  includes  cost,  cost  sharing,  cost-plus- 
fixed-fee,  cost-plus-incentive-fee  contracts,  and  cost-plus-award- 
fee.  A  cost-reimbursement  contract  establishes  an  estimate  of  total 
cost  for  the  purpose  of  obligation  which  the  contractor  may  not 
exceed,  except  at  his  own  risk,  without  prior  approval  or 
subsequent  ratification  of  the  contracting  officer. 

In  general,  a  category  of  contracts  whose  use  is  based  on  the 
establishment  of  a  firm  price  to  complete  the  required  work.  This 
category  includes  firm  fixed-price,  fixed-price  with  escalation, 
fixed-price  redeterminable,  and  fixed-price  with  incentive 
provisions  contracts. 

Used  when  the  precise  quantity  of  items  or  specific  time  of  delivery 
desired  is  not  known.  Usually  will  specify  a  maximum  and/or 
minimum  quantity.  Such  procurement  is  effected  via  a  definite 
quantity  contract,  a  requirements  contract,  or  an  indefinite 
quantity  contract.  May  be  either  negotiated  or  sealed  bid. 

An  interim  type  of  contractual  agreement,  sometimes  called  a 
“Letter  of  Intent,”  authorizing  the  commencement  of 
manufacture  of  supplies  or  performance  of  services.  Used  in 
negotiated  procurements  only  when  a  definite  fixed-price  or  cost- 
reimbursement  contract  cannot  be  written  until  a  later  date. 

In  general,  a  category  of  contracts  designed  to  facilitate  a 
procurement  for  which  one  of  the  fixed-price  or  cost- 
reimbursement-type  contracts  is  considered  inappropriate.  This 
category  includes  basic  agreements,  indefinite  delivery  type 
contracts,  letter  contracts,  and  time  and  materials/labor  hour 
contracts. 

A  CPFF  contract  may  take  one  of  two  basic  forms:  completion 
or  term.  Completion  describes  the  scope  of  work  by  stating  a 
definite  goal  or  target  and  specifying  an  end  product.  Normally 
requires  the  contractor  to  complete  and  deliver  the  specified  end 
product  within  estimated  cost  as  a  condition  for  payment  of  entire 
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h.  Time  and 
Materials 

Contracting  Officer 

Contracting  Officer’s 

Technical  Representative 
(COTR) 


Contractor 

Cost  Overrun 

DARPA 

Data 

DCAA 

Defense  Contract 

Administration  Service 
(DC  AS) 

a.  DCASMA 

b.  DCASPRO 

c.  DCASR 
DCP 

DDRE 

Delivery  Order 


Deputy  for  Small  Business 


fixed  fee.  Term  describes  the  scope  of  work  in  general  terms  and 
obligates  the  contractor  to  devote  a  specified  level  of  effort  for 
a  stated  time  period. 

Negotiated  contracts  based  on  specified  fixed  hourly  rates  to 
complete  a  given  task.  Used  only  in  situations  where  it  is  not 
possible  at  the  outset  to  estimate  the  extent  or  duration  of  the  work 
involved  or  to  anticipate  cost  with  any  substantial  accuracy. 

Government  official  who  by  position  or  appointment  is  authorized 
to  bind  the  government  in  contracts  as  an  agent  for  the 
government. 

An  individual  appointed  in  writing  by  the  commanding  officer 
of  the  requiring  activity,  or  the  contracting  officer,  with  duties 
assigned  by  the  contracting  officer,  who  functions  as  the  technical 
representative  of  the  contracting  officer  in  the  administration  of 
a  specific  contract  or  delivery  order.  The  COTRs  general  duties 
include  providing  technical  direction  as  necessary  and  authorized 
with  respect  to  the  specifications  or  statement  of  work  and 
monitoring  the  progress  and  quality  of  contractor  performance. 

Private,  nongovernment  party  who  enters  into  a  contract  with  the 
government.  Vendor  is  a  term  also  used  to  denote  contractor. 

The  amount  by  which  a  contractor  exceeds  the  estimated  cost 
and/or  the  final  limitation  (ceiling)  of  his  contract. 

Defense  Advanced  Research  Projects  Agency 

All  recorded  information  to  be  delivered  under  a  contract. 
“Technical  data”  excludes  management  and  financial  data. 

Defense  Contract  Audit  Agency 

An  agency,  under  direction  of  Director  of  DSA,  to  provide  unified 
contract  administration  services  to  DoD  components  and  NASA 
for  all  contracts,  except  those  specifically  exempted. 

Defense  Contract  Administration  Services  Management  Area 

Defense  Contract  Administration  Services  Plant  Representative 
Office 

Defense  Contract  Administration  Services  Region 
Development  concept  paper;  decision  coordination  paper 
See  ODDR&E 

A  contractual  document  on  DD  form  1 155  issued  by  a  contracting 
or  ordering  officer  under  an  existing  contract.  A  delivery  order 
automatically  incorporates  all  the  terms  and  conditions  in  the  basic- 
contract. 

At  NOSC,  Code  202;  previously  the  small  and  disadvantaged 
business  utilization  specialist  (SADBUS) 
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Determination  and  Findings 
(D&F) 


DFARS 


DIPEC 
Direct  Cost 


DNFYP 


DNSARC 


DRRB 

DSARC 


Expressly  Unallowable  Cost 


Federal  Acquisition 
Regulation  (FAR) 


F  AD 


A  special  form  of  written  approval  by  an  authorized  official  that 
is  required  by  statute  or  regulation  as  a  prerequisite  to  taking 
certain  contracting  actions.  The  determination  is  a  conclusion  or 
decision  supported  by  the  findings.  Examples  are  making  advance 
payments  in  negotiated  acquisitions  and  determining  the  type  of 
contract  to  use. 

DoD  FAR  supplement 

Data  item  description  (DD  1664) 

Defense  Industrial  Plant  Equipment  Center 

Any  cost  which  is  identified  specifically  with  a  particular  final  cost 
objective.  Not  limited  to  items  which  are  incorporated  in  the  end 
product  as  material  or  labor. 

Defense  Logistics  Agency 

Department  of  the  Navy  Five-Year  Plan 

Director  of  Navy  Laboratories 

Department  of  the  Navy  Systems  Acquisition  Review  Council 

Delegation  of  procurement  authority 

Data  Requirements  Review  Board 

Defense  Systems  Acquisition  Review  Council 

Design  to  cost 

Engineering  change  proposal 

Particular  item  or  type  of  cost  which,  under  the  express  provisions 
of  an  applicable  law,  regulation,  or  contract,  is  specifically  named 
and  stated  to  be  unallowable. 

The  basic  regulation  for  the  conduct  of  government  acquisition. 
Further  implemented  by  departmental  acquisition  instructions. 

An  amount,  in  addition  to  allowable  costs,  paid  to  contractors 
having  cost-plus-fixed- fee  or  cost-plus-incentive-fee  contracts.  In 
cost-plus-fixed-fee  contracts  the  fee  is  fixed  as  a  percentage  (stated 
in  a  dollar  amount)  of  the  initially  estimated  cost  of  the  acquisition. 
In  cost-plus-incentive- fee  contracts  the  fee  is  expressed  in  a 
maximum  and  minimum  amount,  along  with  a  fee  adjustment 
formula  that  provides  the  incentive  for  a  reduction  in  cost  to  the 
government.  Statutory  limitations  are  prescribed  for  the  maximum 
setting  of  fees. 

Force/activity  designator 

Financial  management  advisor 

Free-on-board  or  freight-on-board 

Federal  supply  class 


r 

'  ’  ’  7  ^  r  r  r  '  -  v  -l‘l  i.\  1  K  f  A  ’ 

*  » 

FV  DP 

Full  &  Open  Competition 

Five-year  defense  program 

All  responsible  sources  are  permitted  to  compete. 

l 

P 

General  Accounting  Office 
(GAO) 

An  agency  of  the  legislative  branch,  responsible  solely  to  the 
Congress,  which  functions  to  audit  all  negotiated  government 
contracts  and  investigate  all  matters  relating  to  the  receipt, 
disbursement,  and  application  of  public  funds.  Determines  whether 
public  funds  are  expended  in  accordance  with  appropriations  and 
law. 

1 

• 

» 

General  and  Administrative 
(G&A) 

Any  management,  financial,  and  other  expense  which  is  incurred 
by  or  allocated  to  a  business  unit  and  which  is  for  the  general 
management  and  administration  of  the  business  unit  as  a  whole. 
Does  not  include  those  management  expenses  whose  beneficial 
or  causal  relationship  to  cost  objectives  can  be  more  directly 
measured  by  a  base  other  than  a  cost  input  base  representing  the 
total  activity  of  a  business  unit  during  a  cost  accounting  period. 

► 

« 

s 

N 

> 

> 

• 

General  Provisions 

The  mandatory  and  applicable  (by  law  or  regulation)  clauses  for 
all  DoD  contracts  for  the  type  of  acquisition  involved  —  sometimes 
called  “boiler  plate.”  The  clauses  devised  particularly  for  the 
acquisition  are  called  the  special  provisions. 

4 

» 

i  v. 

Government-Finished 
Property  (GFP) 

Government-owned  property  furnished  to  a  contractor  for  the 
performance  of  a  contract.  Defined  as  industrial  facilities,  material, 
special  tooling,  special  test  equipment,  military  property.  Also 
designated  government-furnished  material  (GFM),  government- 
furnished  equipment  (GFE),  government-furnished  information 
(GF1),  and  government-furnished  facility  (GFF). 

» 

GSA 

General  Services  Administration 

V 

V 

HCA 

Head  of  contracting  agency 

, 

IGCE 

Independent  government  cost  estimate 

* 

ILS 

Integrated  logistic  support 

Indirect  Cost 

Any  cost  not  directly  identified  with  a  single  final  cost  objective, 
but  identified  with  two  or  more  final  cost  objectives  or  with  at 
least  one  intermediate  cost  objective. 

IPD 

Issue  priority  designator 

1 

* 

IPF 

Industrial  plant  equipment  (See  DIPEC) 

• 

IR&D 

Independent  research  and  development 

* 

1R  IFD 

Independent  research  independent  exploratory  development 

j 

Invitation  for  Bids  (IFB) 

The  solicitation  form  used  in  sealed  bid  acquisitions  and  in  step 
two  of  two-step  advertising  (see  below).  All  sealed  bid  acquisitions 
must  be  on  invitation  for  bids. 

, 

• 

,i 

K 

Contract 

j 

4 

*  . 

(  \ 

M 

• 

KO 

Contracting  officer  (see  also  CO) 
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J 

J 

j 
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\  j 

KR 

ICC 

LSA 

Master  (Delivery  Order) 
Contract 


MIC 

MIPR 

MYP 

NARSUP 

NAVSUP 

NICRAD 

NRCC 

NSN 

ODDR&E 

Offer/Proposal/Quotation 

Option 

Ordering  Officer 

Preaward  Survey  (Facility 
Capability  Review  — 
FCR) 

Preproposal  Conference 


Presolicitation  Conference 


Contractor 
Life-cycle  costing 
Labor  surplus  area 

A  type  of  agreement  describing  the  total  desired  area  of  contractor 
performance  and  breaking  this  down  into  a  number  of  broadly 
defined  tasks.  The  contractor  is  obligated  to  perform  delivery 
orders  subsequently  issued  by  the  government  under  the  terms  and 
conditions  in  the  master  contract. 

Management  Information  Center 

Military  indepartmental  procurement  request 

Multiyear  procurement 

Navy  Acquisition  Regulation  Supplement 

Naval  Supply  Systems  Command 

Navy/Industry  Cooperative  R&D  Program 

Naval  Regional  Contracting  Center 

National  stock  number 

Office  of  the  Director,  Defense  Research  and  Engineering 

A  prospective  contractor’s  response  to  the  solicitation  form 
(RFP/RFQ/IFB). 

A  contractual  clause  permitting  an  increase  in  the  quantity  of 
supplies  beyond  that  originally  stipulated  or  an  extension  in  the 
time  for  which  services  on  a  time  basis  may  be  required. 

An  individual  appointed  by  the  Commanding  Officer  of  NOSC 
and  authorized  by  warrant  to  issue  delivery  orders. 

Study  of  a  prospective  contractor’s  financial,  organizational,  and 
operational  status  made  prior  to  contract  award  to  determine  his 
responsibility  and  eligibility  for  government  procurement. 

A  meeting  held  with  potential  contractors  a  few  days  after  requests 
for  proposals/quotes  or  invitation  for  bids  have  been  sent  out, 
to  promote  uniform  interpretation  of  work  statements  and 
specifications  by  all  prospective  contractors. 

A  meeting  held  with  potential  contractors  prior  to  a  formal 
solicitation,  to  discuss  technical  and  other  problems  connected 
with  a  proposed  procurement.  The  conference  is  also  used  to  elicit 
the  interest  of  prospective  contractors  in  pursuing  the  task. 


Price  and  Fee 

a.  Ceiling  Price 


The  monetary  limit  in  a  fixed-price  type  contract  that  the 
government  is  obligated  to  pay.  Costs  incurred  beyond  this  point 
must  be  absorbed  by  the  contractor. 


b.  Target  Price 

Procurement  Request  (PR) 


Procuring  Contracting 
Officer  (PCO) 


Purchase  Order  (PO) 


QA 

QC 

QRC 

Qualified  Products  List 
(QPL) 

RDC 

R&D 

RD&E 

RDT&E 

Request  for  Contractual 
Procurement  (RCP) 

Request  for  Proposal  (RFP) 


Request  for  Quotation 
(RFQ) 


The  estimate  of  price  —  in  a  fixed-price  redeterminable  or  incentive 
contract  —  that  the  government  expects  to  pay  for  supplies 
procured  under  the  contract. 

Document  which  describes  the  required  supplies  or  services  so  that 
a  procurement  can  be  initiated.  Some  procuring  activities  actually 
refer  to  the  document  by  this  title,  others  use  different  titles,  such 
as  procurement  directive,  and  so  forth.  At  NOSC,  it  is  termed 
the  stub  requisition  or  acquisition  requirements  package  (ARP). 

The  government  contracting  officer  directing  and  administering 
the  procurement  through  the  award  of  the  contract  and  the  signing 
of  the  actual  contractual  documents.  Administration  of  the 
contract  after  award  may  be  delegated  to  an  ACO,  as  prescribed 
above. 

A  contractual  procurement  document  used  primarily  to  procure 
supplies  and  nonpersonal  services  when  the  aggregate  amount 
involved  in  any  one  transaction  is  relatively  small  (for  example, 
not  exceeding  $25,000).  (DD  form  1155.) 

Quality  assurance 

Quality  control 

Quick-reaction  capability 

A  list  of  products  which  are  pretested  in  advance  of  actual 
procurement  to  determine  which  suppliers  can  comply  properly 
with  specification  requirements.  This  is  most  usually  done  because 
of  the  length  of  time  required  for  test  and  evaluation. 

Rapid  development  capability 

Research  and  development 

Research,  development,  and  engineering 

Research,  development,  test,  and  evaluation 


Document  which  describes  the  required  supplies  or  services  so  that 
an  acquisition  can  be  initiated.  Some  contracting  activities  actually 
refer  to  the  document  by  this  title,  others  use  different  titles,  such 
as  acquisition,  directive,  and  so  forth. 

The  solicitation  form  used  when  the  governmeni  reserves  the  right 
to  award  without  further  oral  or  written  negotiation.  Only  the 
acceptance  of  the  government  is  required  for  award.  Requests  for 
proposal  are  for  negotiated  acquisitions. 

The  solicitation  form  used  to  obtain  price,  cost,  delivery,  and  other 
information  from  prospective  suppliers  This  procedure,  utilizing 
SF-18  [Request  for  Quotation],  is  authorized  in  both  formal 
contracting  and  informal  contracting;  however,  its  use  is  generally 


Request  for  Technical 
Proposal  (RFTP) 

Responsive  and  Responsible 
Bidder 

Sealed  Bid 

Small  Purchases 

Sole  Source 

SPAWARS 

Specifications 

SSA 

SSEB 

Subcontractor 

SUPARS 

Supplemental  Agreement 

Target  Price 


reserved  for  acquisitions  estimated  not  to  exceed  525,000.  The 
procedure  is  also  used  for  solicitation  of  information  for  planning 
purposes. 

The  solicitation  form  used  to  request  proposals.  Does  not  include 
a  cost  proposal  in  the  first  step  of  a  “two-step  sealed  bid” 
acquisition. 

A  bidder  is  responsive  if  his  bid/proposal  conforms  to  the 
requirements  of  the  IFB/RFP/RFQ,  and  is  responsible  if  he  has 
the  capacity  and  facilities  to  produce  the  supplies  or  render  the 
services  being  procured. 

For  government  acquisition  of  supplies  and  services.  After  public 
opening  of  sealed  competitive  bids,  award  is  made  to  the  lowest 
responsive  and  responsible  bidder,  price  and  other  factors 
considered,  in  accordance  with  FAR  part  14. 

Open-market  buy  of  $25,000  or  less;  simplified  purchases.  This 
category  includes  blanket  purchase  agreement  (BPA)  calls, 
purchase  orders,  imprest  fund  purchases  (cash),  and  standard  form 
44  purchases. 

Contract  entered  into  after  soliciting  and  negotiating  with  only 
one  source. 

Space  and  Warefare  Systems 

Description  of  technical  requirements  for  a  material,  product,  or 
service  that  includes  criteria  for  determining  whether  these  require¬ 
ments  are  met.  Specifications  shall  state  only  the  government’s 
minimum  needs. 

Source  Selection  authority 

Source  Selection  Evaluation  Board 

Private  party  who  enters  into  a  subcontract  with  the  government  's 
contractor.  Generally,  government  is  not  “privy”  to  a  contractor- 
subcontractor  relationship. 

Supply  Acquisition  Regulation  Supplement  (also  known  as 
NAVSUP  Manual  P-560). 

Bilateral  written  amendment  to  a  contract  by  which  the  govern¬ 
ment  and  the  contractor  settle  price  and/or  performance  adjust¬ 
ments  or  alter  any  of  the  terms  and  conditions  of  the  basic  contract. 
See  also  Change  Order  and  Modifications. 

The  estimate  of  price  —  in  a  fixed-price  redeterminable  or  incentive 
contract  —  that  the  government  expects  to  pay  for  supplies 
procured  under  the  contract.  Adjustment  of  the  target  price,  also 
referred  to  as  price  ceiling,  is  made  upon  the  occurrence  of  a  stated 
event  or  contingency  or  by  operation  of  a  contract  clause  or 
provision. 
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Termination 

TCO 
TEMS 
T  for  C 
T  for  D 

Two-Step  Sealed  Bidding 


R  i* 


Unauthorized  Commitment 


UMMIPS 

UND 

Work  Breakdown  Structure 
(WBS) 


ZBB 


The  canceling  of  all  or  a  part  of  the  prime  or  subcontract  prior 
to  its  completion  through  performance.  May  be  for  either  default 
or  convenience. 

Termination  Contracting  Officer 
Technical  Equipment  Management  System 
Termination  for  convenience 
Termination  for  default 

Technical  proposals  (see  RFTP  under  Sealed  Bid)  from  contractors 
are  solicited  in  the  first  step;  after  selection  of  acceptable  proposals, 
each  offeror  submits  a  separate  bid  (second  step),  pricing  his  own 
technical  proposal.  Contract  is  awarded  to  lowest  bidder  in  second 
step.  The  resultant  contract  must  be  a  firm  fixed-price  or  fixed- 
price  with  economic  price  adjustment. 

A  change  or  request  to  perform  by  a  government  official  which 
a  contractor  believes  to  have  apparent  authority,  but  which  is  not 
legally  binding  because  the  government  official  is  not  authorized 
to  commit  the  Federal  government. 

Uniform  Material  Movement  and  Issue  Priority  System 

Urgency  need  designator  (aka:  priority  designator). 

A  framework  that  provides  uniform  approach  to  structuring  the 
program  throughout  the  acquisition  life-cycle  phases.  The  WBS 
permits  a  logical  arrangement  of  the  elements  of  the  statement 
of  work  (SOW)  and  a  tracing  of  work  effort  expended  under  each 
of  the  elements.  (MIL-STD-881  defines  the  WBS  used  for  system 
acquisitions.) 

Zero  base  budgeting 


15.3  CONTRACTING  CONDITIONS  AND  ENVIRONMENT 


15.3.1  Legislation  Affecting  Acquisition 


Three  recent  laws  passed  by  the  98th  Congress  have  had  a  profound  effect  upon  the  federal 
contracting  process: 

The  Competition  in  Contracting  Act  (CICA)  of  1984,  P.L.  98-369  (Title  VII),  effective 
1  April  1985 

The  Small  Business  and  Federal  Procurement  Competition  Enhancement  Act  of  1984, 

P.L.  98-577,  enacted  30  October  1984 

The  Defense  Procurement  Reform  Act  of  1984,  P.L.  98-525,  enacted  19  October  1984. 
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Because  of  the  great  significance  of  this  legislation,  every  person  involved  in  the  federal  contracting 
process  must  have  a  working  knowledge  of  the  contents  of  these  laws.  This  applies  not  only  to  contracting 
specialists  but  also  to  program  managers,  executives,  and  other  decision  makers. 

The  statutory  emphasis  has  shifted  from  the  method  of  procurement  to  the  use  of  sources.  No 
longer  is  how  you  procure  the  principal  matter  of  the  law;  rather  it  is  from  whom  you  procure  that 
is  the  foremost  concern.  In  the  past  the  law  stated  preference  for  the  “formal  advertising’’  method 
over  the  “negotiated”  method.  Law  now  emphasizes  competitive  acquisition  from  among  multiple 
sources  over  acquisition  from  single  sources.  In  other  words,  the  key  consideration  confronting  contracts 
managers  today  is  not  “Is  there  authority  to  negotiate?”  but  “Can  this  acquisition  be  made  on  a 
competitive  basis?” 

Prior  to  the  passage  of  CICA,  all  federal  acquisitions  were  formally  advertised  unless  it  was 
determined  that  one  of  17  exceptions  permitting  negotiation  applied.  Despite  this  preference  for  “formal 
advertising,”  over  “negotiation,”  the  reality  was  that  most  federal  contracts  were  negotiated  —  often 
noncompetitively. 

CICA  did  away  with  the  statutory  emphasis  on  formal  advertising  by  recognizing  “sealed  bidding” 
and  “negotiation”  as  equivalent  competitive  procedures.  CICA  provides  for  only  seven  circumstances 
in  which  “other  than  full  and  open  competition”  is  permitted.  Congress  sought  to  change  the  emphasis 
from  formal  advertising  versus  negotiated  to  “full  and  open  competition”  versus  “other  than  full  and 
open  competition.” 

The  seven  circumstances  allowing  other  than  full  and  open  competition  are  as  follows: 

a.  The  property  or  service  is  available  from  only  one  source  and  no  alternative  product  or  service 
will  meet  the  government’s  need 

b.  An  unusual  and  compelling  urgency  exists 

c.  Procurement  from  a  specified  source  is  essential  to  maintain  a  source  of  supply  in  case  of 
national  emergency,  or  to  establish  or  maintain  essential  R&D  capability 

d.  The  procurement  is  made  under  the  terms  of  an  international  agreement 

e.  The  procurement  is  from  a  source  or  agency  required  by  statute 

f.  Disclosure  of  the  procurement  would  compromise  national  security 

g.  The  head  of  an  agency  determines  in  writing  that  using  other  than  competitive  procedures 
is  “necessary”  and  “in  the  public  interest.” 

The  Small  Business  and  Federal  Procurement  Enhancement  Act  of  1984  is  intended  to  eliminate 
acquisition  procedures  and  practices  that  inhibit  free  and  open  competition  and  to  foster  opportunities 
for  small  business  to  participate  in  competitive  acquisition. 

Finally,  the  Defense  Procurement  Reform  Act  of  1984  is  an  amalgamation  of  congressional  initiatives 
to  improve  the  effectiveness  of  Department  of  Defense  acquisition.  The  many  provisions  of  this  act 
further  control  management  of  the  government’s  acquisition  function  from  specifying  the  amount  of 
overhead  a  contractor  may  properly  allocate  to  supplies  to  how  long  a  DoD  program  manager  must 
be  kept  in  his  position. 

Actually,  these  three  laws  contain  far  more  than  the  brief  descriptions  offered  above.  Without 
doubt,  the  “hidden”  administrative  cost  of  implementing  these  laws  will  be  significant,  and  these  laws 
will  continually  influence  the  course  of  federal  acquisition. 
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15.3.2  Deputy  for  Small  Business 


The  mission  of  the  federal  contracting  process  is  to  buy  the  goods  and  services  that  the  government 
needs  to  operate.  The  acquisition  process,  because  of  its  size  and  impact  on  the  nation’s  economy, 
is  also  used  by  the  government  to  carry  out  its  socioeconomic  goals  as  well.  One  of  these  goals  is  to 
foster  the  growth  and  development  of  small  business. 

At  the  Naval  Ocean  Systems  Center,  the  Deputy  for  Small  Business  (Code  202,  x2707)  is  charged 
with  ensuring  small  business  participation  in  our  acquisitions.  Early  coordination  with  the  NOSC  Deputy 
for  Small  Business  will  permit  effective  small  business  planning  and  prevent  potential  delays  in  the 
processing  of  the  contract  requirement. 

The  government  primarily  uses  the  small  business  set-aside  to  help  it  accomplish  small  business 
assistance  objectives.  All  small  purchases  of  $10,000  or  less  are  set-aside  for  small  business  by  law 
and  are  known  as  “small  business-small  purchase  set-asides.”  Small  businesses  may  furnish  any 
domestically  produced  or  manufactured  product  under  a  small  business-small  purchase  set-aside.  When 
it  is  essential  that  a  large  business  be  solicited  directly,  a  brief  justification  is  required  to  dissolve  the 
^et-aside  and  to  document  the  file. 

Requirements  estimated  to  exceed  $10,000,  including  both  small  purchase  and  contract  requirements, 
are  individually  screened  by  the  buyer  or  negotiator  and  the  Deputy  for  Small  Business  for  potential 
small  business  set-aside.  Small  business  set-asides  in  excess  of  $10,000  differ  from  small  business  small 
purchase  set-asides  in  that  the  products  must  also  be  manufactured  by  small  business.  Requirements 
not  set-aside  for  small  business  must  be  justified.  The  justifications  are  subject  to  review  by  the  Small 
Business  Administration  (SBA). 

Other  preferential  programs  for  small  business  include  increased  opportunities  for  subcontracting 
a  special  program  for  disadvantaged  firms.  The  prime  contractor  of  a  contract  over  $10,000  must  agree 
that  small  business  concerns  and  small  disadvantaged  business  concerns  shall  have  the  maximum 
practicable  opportunity  to  participate  in  contract  performance.  Each  solicitation  for  a  contract  expected 
to  exceed  S500.000  that  has  subcontracting  possibilities  must  require  the  offeror  to  submit  an  acceptable 
subcontracting  plan.  Under  Section  8(a)  of  the  Small  Business  Act  disadvantaged  firms  certified  by 
the  Small  Business  Administration  for  entry  into  the  8(a)  program  are  eligible  for  noncompetitive  contracts 
for  federal  contracting  activities.  Section  8(a)  firms  can  provide  a  valuable  resource  for  project  managers. 

Small  business  programs  are  expected  to  continue  throughout  the  remainder  of  the  decade  although 
the  procedures  for  the  programs  may  undergo  some  changes. 

15.3.3  Data  Management  Office 


The  Data  Management  Office  (DMO),  Code  921 1,  serves  as  the  data  management  central  focal 
point  and  advises  and  assists  in  carrying  out  NOSC1NST  4000.1  and  related  directives. 

All  NOSC  acquisitions  for  which  data  is  required,  and  all  NOSC  acquisitions  of  $10,000  or  more 
diall  be  reviewed  by  the  DMO. 

All  acquisition  requirements  packages  (ARPs)  with  an  estimated  cost  of  $1  million  or  more,  or 
ARPs  w  here  the  cost  of  data  is  estimated  at  $100,000  or  more  shall  be  reviewed  by  a  Data  Requirements 
Review  Board  (DRRB).  The  DMO  shall  convene  and  chair  the  DRRB. 

Any  specific  questions  should  be  directed  to  the  Data  Management  Office  (Code  921 1),  x 5 8 1 7  '7038. 
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15.3.4  Contracting  Officer’s  Technical  Representative  Duties  and  Responsibilities 


As  a  Contracting  Officer’s  Technical  Representative  (COTR)  at  NOSC  you  are  charged  with  an 
incredible  responsibility:  to  act  as  technical  liaison  between  the  contractor  and  the  NOSC  Contracts 
Division  without  compromising  the  government’s  contractual  integrity.  It  is  important,  however,  to 
recognize  that  the  acquisition  process  requires  a  team  effort.  Close  coordination  and  cooperation  among 
the  various  team  members  is  very  critical  to  the  success  of  the  acquisition.  The  designated  COTR  provides 
a  key  role  for  much  of  the  effort.  He  or  she  is  responsible  for  principal  tasks  in  each  of  the  three  phases 
of  the  acquisition  process  as  follows: 

a.  Planning  the  technical  requirements  and  providing  much  of  the  documentation  to  form  the 
acquisition  request  package  (ARP) 

b.  Usually  serving  as  chair  of  the  evaluation  team  which  reviews  the  technical  proposals  to 
determine  technical  acceptability 

c.  Managing  the  resultant  contract  by  scrutinizing  technical  performance,  progress,  incurred  costs, 
and  inspection  and  acceptance  of  contract  deliverables. 

Only  an  individual  who  has  received  formal  COTR  training  and  possesses  the  requisite  technical 
skills  and  experience  to  monitor  the  service  contracts  effectively  can  be  appointed  as  a  COTR.  NOSCINST 
4330.1  provides  detailed  information  on  training,  qualifications,  appointment,  duties,  and  responsibilities. 

15.3.5  Unauthorized  Commitments 

An  informal  or  unauthorized  commitment  is  an  action  by  employees  other  than  duly  designated 
procuring  contracting  officers  (PCO)  or  purchasing  agents  that  would  cause  a  supplier  to  deliver  material 
or  service  in  expectation  that  the  service  and/or  materials  will  be  paid  for  from  appropriated  funds. 

At  NOSC,  criminal  and  administrative  penalties  are  provided  for  such  unauthorized  actions.  This 
type  of  informal  or  unauthorized  conduct  does  not  in  fact  obligate  the  government;  however,  it  may 
obligate  the  individual  committing  the  act. 

A  constructive  change  is  an  unauthorized  commitment  for  which  a  contractual  vehicle  exists  under 
which  a  PCO  may  be  able  to  “ratify”  the  commitment,  thereby  obligating  the  government  rather  than 
the  individual. 


15.4  ADVANCE  ACQUISITION  PLANNING  AND  ACQUISITION  PLANS  (APs) 


15.4.1  Background 


The  presolicitation  phase  is  the  basis  for  the  entire  contracting  process.  It  includes  planning  the 
contract  and  consolidating  and  refining  these  plans  into  the  necessary  documentation  and  the  approvals 
that  make  up  the  acquisition  requirements  package  (ARP).  (Appendix  15A  lists  the  items  necessary 
for  inclusion  in  the  ARP.  Appendixes  15B  (concerning  the  stub  requisition  form  and  its  use)  and  15C 
(addressing  NOSC’s  internal  approval  control  points  for  commodities  and  documentation)  further  define 
and  explain  the  stub  requisition  requirement  in  the  ARP.) 
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15.4.2  Benefits  of  Contract  Planning 


Contract  planning  encompasses  a  wide  range  of  considerations  that  affect  the  entire  acquisition 
cycle.  Nothing  is  more  critical  to  the  success  of  the  program  than  the  informed,  reasonable  planning 
of  it.  Planning  ensures  the  best  use  of  funds,  personnel,  networks,  and  other  resources  to  achieve  timely 
delivery  of  quality  supplies  and  services. 

The  importance  of  planning  cannot  be  stated  too  strongly.  During  the  entire  contracting  process, 
adequate  planning  lessens  problems  and  delays.  It  will  prove  to  be  very  important  to  technical  personnel 
who  will  be  monitoring  the  resultant  contract.  The  contract  scope  and  terms  and  conditions  that  are 
determined  at  this  stage  define  the  contractual  rights  and  duties  of  the  contractor  and  the  government. 
These  terms  and  conditions  may  be  difficult  to  change  later  if  some  contractual  aspects  were  overlooked 
and  potential  problems  had  not  been  anticipated. 

Contract  planning  includes  careful  consideration  of  the  technical,  scheduling,  and  cost  aspects 
of  the  contract.  Technical  aspects  involve  defining  the  scope  or  the  technical  boundaries  within  which 
the  tangible  products  (deliverable  items)  will  be  provided  by  the  contractor.  Scheduling  aspects  include 
incorporating  control  points  into  the  contracted  effort  by  which  progress  can  be  measured  and  costs 
can  be  evaluated.  The  data  to  be  delivered  by  the  contractor  should  relate  to  these  control  or  decision 
points.  Cost  aspects  include  preparation  of  a  government  estimate  for  the  effort  to  be  performed.  This 
enhances  the  government’s  ability  to  select  a  contractor  whose  price  is  realistic  and  appropriate.  It 
lessens  the  chances  of  subsequent  overrun  situations  and  of  a  contractor’s  “buying  in”  to  place  himself 
in  an  advantaged  or  sole  source  position  for  future  work. 

Other  planning  considerations  include  selecting  the  most  appropriate  type  of  contract,  screening 
potential  contractors  through  a  sources-sought  synopsis,  determining  the  need  for  government-furnished 
resources,  and  making  necessary  arrangements  for  the  timely  provision  of  these  resources.  If  the  contract 
will  involve  automated  data  processing  (ADP)  services,  appropriate  documentation  must  be  developed, 
reviewed,  and  approved. 

Sound  planning  includes  the  delegation  of  contracting  tasks  and  the  determination  of  definitive 
courses  of  action  to  be  taken  and  deadlines  for  their  completion.  Much  contract  planning  is  accomplished 
through  informal  meetings.  Information  gathered  during  these  meetings  is  consolidated  and  refined 
in  the  SOW  and  additional  planning  documentation  is  often  unnecessary.  If  the  contract  is  unusually 
complex,  has  a  large  dollar  value,  or  covers  an  extended  period  of  performance,  more  formal  planning, 
such  as  PERT  charges  and  work  breakdown  structures,  may  be  desirable.  To  help  set  the  proper  course 
for  the  contract,  contractual  requirements  should  be  discussed  at  an  early  stage  with  the  contracts  branch. 

15.4.3  Timing  Considerations 

Timing  is  a  vital  part  of  good  planning.  After  determining  the  date  the  contracted  effort  is  to 
be  completed  and  the  time  necessary  for  the  contractor  to  perform  the  effort,  it  should  be  possible 
to  ascertain  the  necessary  contract  award  date.  Then,  by  considering  the  average  procurement 
administrative  lead  time  (PALT)  required  to  process  the  contract,  the  date  that  the  acquisition 
requirements  package  must  be  submitted  to  the  contracts  branch  can  be  calculated. 

The  importance  of  beginning  contractual  actions  at  the  proper  time  cannot  be  overemphasized. 
Allowing  adequate  time  will  ensure  that  contractual  services  will  be  available  when  needed.  The  acquisition 
requirements  package  must  be  submitted  to  the  contracts  branch  well  in  advance  of  the  necessary  award 
data  to  provide  time  to  process  the  contract. 


15.4.4  Acquisition  Plans 


Acquisition  planning  is  the  process  by  which  all  personnel  responsible  for  an  acquisition  coordinate 
and  integrate  a  comprehensive  plan  for  fulfilling  the  identified  program  need  promptly  and  reasonably, 
including  the  development  of  an  overall  strategy  for  managing  the  acquisition. 

The  acquisition  plan  (AP),  on  the  other  hand,  is  a  formal  narrative  description  which  transmits 
acquisition  planning  information  in  a  prescribed  format  to  the  Assistant  Secretary  of  the  Navy 
(Shipbuilding  and  Logistic)  for  his  approval.  No  contracting  action  (including  synopsizing)  is  authorized 
until  the  AP  has  been  approved. 

An  AP  shall  be  prepared  for  development  acquisition  whose  total  in-house  and  contractual  cost 
(including  options)  is  estimated  at  $2  million  or  more,  and  for  production  and  service  acquisitions  whose 
contractual  cost  is  estimated  to  be  in  excess  of  $5  million. 

Such  significant  elements  as  dollar  thresholds,  exemptions,  review  cycles,  and  format  requirements 
are  subject  to  change.  Therefore,  the  contracts  branch  which  serves  your  code  should  be  consulted 
for  an  analysis  of  your  requirement  and,  based  on  the  most  current  directives,  the  branch  will  determine 
whether  an  AP  is  required. 

If  ultimately  required,  an  AP  will  be  your  first  step  in  the  formal  acquisition  process.  As  APs 
may  cover  what  is  to  be  acquired  under  one  or  more  contracts,  it  is  desirable  that  you  begin  preparation 
as  soon  as  the  program/acquisition  requirements  can  be  identified.  At  present,  the  average  approval 
time  for  APs  is  100  days.  A  conservative  estimate  for  drafting  an  AP  is  60  to  90  days.  Generally,  an 
AP  should  be  under  review  by  the  Director,  ASN(S&L),  at  least  6  months  prior  to  the  targeted  solicitation 
release.  Remember,  only  after  the  AP  is  approved  and  returned  to  NOSC  can  the  contracts  branch 
initiate  any  actions  on  vour  acquisition  requirements  package  (ARP).  To  prevent  a  return  of  your  AP 
for  revision,  seek  the  advice  and  use  the  expertise  of  the  contracts  branch  personnel  who  can  provide 
clarification  and  guidance  as  you  prepare  the  AP. 

Due  to  recent  scrutiny  of  the  government’s  acquisition  process,  APs  will  be  an  absolute  requirement 
for  the  foreseeable  future.  Presently,  the  AP  is  the  principal  document  for  program  review  and  oversight. 
You  must  develop,  in  the  prescribed  format,  a  succinct  and  cogent  overview  of  your  program  plan. 
Provide  evidence  to  demonstrate  a  cohesive  strategy  for  acquiring  and  managing  the  supplies  or  services, 
and  justify  their  necessity  and  impact  throughout  the  life  cycle  of  the  plan.  In  summary,  the  AP  is 
your  best  effort  to  define  and  explain  your  requirement  and  to  demonstrate  the  thoughtful  planning 
that  went  into  it. 

Once  th'-  AP  is  approved,  it  must  be  reviewed  for  changes  at  least  once  a  year  or  at  milestones 
specified  in  the  AP  or  whenever  significant  changes  occur.  Revisions  shall  be  submitted  to  the  contracts 
branch  for  approval  by  ASN(SAL). 

As  the  AP  contains  acquisition  sensitive  information,  each  page  shall  be  marked  “For  Official 
I  se  Only,”  and  everyone  associated  with  the  requirement  shall  be  counseled  that  information  contained 
in  the  AP  is  not  to  be  disclosed. 

To  conclude,  specific  guidance  and  detailed  information  concerning  preparation,  routing,  and  format 
of  your  AP  can  be  obtained  from  the  contracts  branch.  Coordinate  with  them  early  to  prevent  costly 
delays  which  may  jeopardize  your  requirement. 


15.5  FURTHER  PRESOLICITATION  PREPARATION 


15.5.1  Statement  of  Work 

A  clear  statement  of  contract  requirement  is  a  prerequisite  for  defining  and  achieving  program 
goals.  The  statement  of  work  (SOW)  provides  the  basic  framework  for  this  effort.  As  such,  the  SOW 
must  be  carefully  constructed  to  specify  basic  responsibilities  and  minimum  program  requirements. 
The  SOW  is  a  dynamic  document  established  to  tailor  cost  drivers  based  upon  the  needs  and  limitations 
of  each  acquisition.  The  SOW  writer  must  ensure  that  the  technical  requirements  are  equated  to  those 
minimal  needs. 

Care  and  skill  exercised  in  the  preparation  of  the  SOW  can  be  of  great  significance  by  establishing 
a  conclusive  baseline  upon  which  proposal  evaluation  criteria  can  be  constructed.  Furthermore,  benefits 
resulting  from  a  definitive  SOW  should  result  in  conclusive  proposals  and  reduce  the  time  for  evaluation. 

In  the  actual  proposal  evaluation  and  contractor  selection,  the  SOW  plays  a  significant  role.  Failure 
to  describe  the  scope  of  work  adequately  will  result  in  needless  delays  and  extra  administrative  effort 
during  the  source  selection  process.  The  ability  to  define  the  desired  end  work  products  clearly  and 
in  an  exacting  manner  generally  will  spell  the  difference  in  the  type  of  procurement  approach  that  will 
be  taken  and  the  type  contract  awarded. 

The  role  of  the  SOW  is  to  define  those  work  tasks  which  cannot  be  contained  in  a  specification 
and  must  never  be  included  in  the  Contract  Data  Requirements  List  (CDRL)  or  Data  Item  Description 
(DID).  When  properly  written,  the  SOW  establishes  nonspecification  tasks  and  identifies  the  work  effort 
to  he  performed  expressed  as  minimal  needs.  As  the  contractor  performs  the  effort  and  completes  the 
tasks,  information  that  may  be  required  for  retention  will  be  inherently  developed  with  the  work 
performed.  The  CDRL  is  used  only  to  list  and  order  the  contract  data  required,  while  the  DID  is  used 
to  describe  the  data  and  prescribe  the  preparation  instructions  in  terms  of  format  and  arrangement. 

The  SOW  must  not  be  used  to  amend  the  equipment/system  acquisition  contract  specification, 
and  it  does  not 

Order  data 

Describe  data 

Discuss  data 

Invoke,  cite,  or  discuss  a  DID 
Discuss  a  CDRL 

Specify  design  control  parameters  or  the  performance  of  hardware 
Specify  technical  proposal  criteria  or  evaluation  factors 

Establish  a  delivery  schedule  but  may  include,  for  clarity,  significant  milestones 

Invoke  military  standards  or  specifications  unless  all  facets  are  required  to  meet  minimal  needs 

Invoke  in-house  management,  DoD,  or  departmental  instructions 

Use  data  words  or  identity  the  deliverable  data  to  describe  the  works  to  be  accomplished 
However,  the  optimum  statement  of  work 


Defines  clearly  all  nonspecification  requirements  and  task  efforts  where  conclusive  design  or 
performance  limitations  may  be  expressed  for  needs  beyond  the  objectives  and  goals 

Employs  work  words  to  describe  explicitly  and  in  exacting  terms  what  tasks  shall  be  accomplished 

Provides  a  priceable  or  cost  estimatable  set  of  tasks  to  fulfill  the  government’s  minimum 

States  clearly  and  fully  what  is  required  to  satisfy  the  contract,  but  it  does  NOT  over  specify  nor 
does  it  tell  the  offeror  exactly  how  to  do  it. 

15.5.2  Specifications 

Depending  upon  the  nature  of  the  acquisition,  specifications  may  be  required.  Specification  means 
a  description  of  the  technical  requirements  for  a  material,  product,  or  service  that  includes  the  criteria 
for  determining  whether  these  requirements  are  met.  Specifications  shall  state  only  the  government’s 
actual  minimum  needs  in  a  manner  to  encourage  maximum  practicable  competition. 

Items  to  be  acquired  shall  be  described  by  citing  the  applicable  specifications  and  standards  or 
by  a  description  containing  the  necessary  requirements.  Specifications  and  standards  shall  be  selectively 
applied  and  tailored  in  their  application.  “Selective  application”  is  the  process  of  reviewing  and  selecting 
from  available  specifications,  standards,  and  related  documents  those  particulars  which  have  application 
to  a  specific  acquisition.  “Tailoring”  is  the  process  by  which  individual  sections,  paragraphs,  or  sentences 
of  the  selected  specifications,  standards,  and  related  documents  are  reviewed  and  modified  so  that  each 
one  selected  states  only  the  government’s  minimum  requirements. 

Unless  otherwise  authorized  by  law  or  by  accepted  deviation  (see  FAR,  Part  10.007),  specifications 
and  standards  listed  in  the  Index  of  Federal  Specifications  and  Standards  are  mandatory  for  use  by 
ali  agencies  acquiring  supplies  or  services  covered  by  such  specifications  and  standards.  The  other  principal 
document  covering  this  subject  is  the  Department  of  Defense  Index  of  Specifications  and  Standards 
(DoDISS).  (Both  the  aforementioned  documents  may  be  purchased  from  the  Superintendent  of 
Documents,  U.S.  Government  Printing  Office,  Washington,  D.C.  20402.) 

The  number  and  title  format  for  preparing  a  specification  is  prescribed  as  follows: 

1.  SCOPE 

2.  APPLICABLE  DOCUMENTS 

3.  REQUIREMENTS 

4.  QUALITY  ASSURANCE  PROVISIONS 

5.  PREPARATION  FOR  DELIVERY 

6.  NOTES 

10.  APPENDIX 

Subject  matter  shall  be  kept  within  the  scope  of  the  sections  so  that  the  same  kind  of  requirement 
for  information  will  always  appear  in  the  same  section  of  every  specification.  Except  for  appendixes, 
if  there  is  no  information  pertinent  to  a  section,  the  following  shall  appear  below  the  section  heading: 

“This  section  is  not  applicable  to  this  specification.” 

The  paramount  consideration  in  a  specification  is  its  technical  essence,  and  this  should  be  presented 
in  language  free  of  vague  and  ambiguous  terms.  Using  the  simplest  words  and  phases  will  convey  the 
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intended  meaning.  Inclusion  of  essential  information  shall  be  complete,  whether  by  direct  statements 
or  reference  to  other  documents.  Sentences  shall  be  as  short  and  concise  as  possible.  See  M1L-STD-490 
for  an  elaboration  of  this  subject. 


15.5.3  Technical  Evaluation  Plan 

The  process  of  evaluating  offers  is  unique  to  the  negotiated  method  of  acquisition.  This  method 
allows  award  based  on  price  and  other  factors.  In  sharp  contrast,  sealed  bidding  requires  an  agency 
to  award  the  contract  to  the  lowest  responsible,  responsive  bidder. 

As  stated  in  FAR  15.605,  evaluation  criteria  should  be  individually  tailored  to  each  solicitation 
and  should  reflect  the  requirements  minimum  needs,  yet  not  be  so  restrictive  as  to  limit  competition. 
When  evaluation  criteria  are  properly  selected  and  weighted,  the  evaluation  process  is  simplified. 

The  purpose  of  the  evaluation  is  to  select  the  source(s)  whose  proposal  has  the  highest  degree  of 
realism  and  credibility,  and  whose  performance  is  expected  to  meet  the  government  objectives  best 
at  an  affordable  cost.  The  evaluation  is  to  be  accomplished  in  a  manner  which  assures  an  impartial, 
equitable,  and  comprehensive  evaluation  of  each  offeror’s  proposal  and  related  capabilities  and  which 
minimizes  the  complexity  of  the  solicitation  while  maximizing  the  efficiency  of  the  evaluation/ selection 
decision. 

The  evaluation  process  includes  the  actual  evaluation,  negotiations  to  clarify  details,  and  source 
selection.  The  evaluation  plan  itself  is  formulated  prior  to  the  solicitation  and  serves  the  following 
purposes: 

To  ensure  that  all  efforts  are  directed  toward  a  common  goal 

To  collect,  organize,  and  display  the  performance,  schedule,  and  cost  requirements  by  emphasizing 
pertinent  evaluation  criteria 

To  provide  a  structure  for  oiganizing  the  evaluation  group  and  scheduling  its  activities 

To  provide  a  structure  for  the  preparation  of  the  RFP 

To  establish  a  format  for  discussion  at  preproposal  conferences,  if  held,  and  later  offeror  or 
contractor  discussions 

To  serve  as  a  guide  for  the  contracting  authority  in  source  selection 

To  provide  procedures  and  methodology  for  evaluation  purposes 

To  provide  guidelines  for  making  trade-offs  among  and  within  the  various  factors  to  the  performance 
of  the  equipment  and  to  the  management  of  the  project  in  relationship  to  the  development, 
production,  operating  and  support  costs,  the  delivery  schedule  and  quantity,  and  the  qualitative 
requirements  of  the  procurement. 

The  actual  evaluation  is  based  upon  the  scoring  of  each  proposal  against  the  preestablished  evaluation 
criteria  ranked  in  order  of  importance  and  weighted  accordingly.  Specific  factors  and  rankings  will 
vary  with  each  procurement,  but  the  cost  proposal  will  always  be  evaluated  separately  from  the  other  part. 

The  government  evaluates  each  proposal  against  its  own  previous  estimates  to  assess  the  realism 
of  cost,  schedule,  risk  assessment,  and  technical  approach  and  against  established  standards  for 
management,  accounting  practices,  and  the  like.  Proposals  which  are  unrealistically  low  in  cost  or  price 
can  be  rejected  on  the  grounds  that  the  offeror  fails  to  comprehend  the  complexities  and  risks  of  the 
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contract  requirements  or  else  has  made  an  improvident  proposal.  Only  proposals  which  are  evaluated 
as  acceptable  (by  preestablished  criteria)  will  be  passed  for  final  source  selection. 

The  final  source  selection  is  an  integrated  decision  based  on  a  consideration  of  technical  approach, 
capability,  management,  design-to-cost,  historical  performance,  price/cost,  and  other  pertinent  factors. 
The  selected  source(s)  should  the  one(s)  who  is  expected  to  do  the  best  overall  job  for  the  government. 
The  selected  offeror’s  proposal  (technical,  management,  and  cost)  must  satisfy  the  government’s 
minimum  requirement. 


15.6  OTHER  THAN  FULL  AND  OPEN  COMPETITION 


15.6.1  Sole  Source  Justification  and  Justification  and  Authorization 

Both  the  sole  source  justification  (SSJ)  (NOSC-SD  4200/5)  and  the  justification  and  authorization 
(J&A)  shall  be  prepared  for  acquisitions  for  other  than  full  and  open  competition.  The  J&A  shall  be 
prepared  by  the  contracts  branch  and  the  SSJ  by  the  requester. 

'  /ritten  justification  is  not  required  for  acquisitions  under  $1,000;  however,  when  the  requiring 
code  has  knowledge  that  competition  exists,  this  information  shall  be  included  in  the  request. 

All  requests  for  sole  source  acquisitions  of  $1,000  or  more  will  be  accompanied  by  the  SSJ,  Stub 
(NOSC-SD  4235/4),  SOW/specifications,  and  all  other  pertinent  information  including  appropriate 
approvals  within  the  department.  In  all  cases,  SSJs  will  be  signed  by  the  originator,  reviewed,  and 
approved  by  the  cognizant  division  head  up  to  $50,000  (the  threshold  will  be  raised  to  $100,000  soon) 
and  approved  by  the  appropriate  department  head  (who  must  be  SES  level)  if  $50,000  ($100,000  soon) 
or  over. 

Sole  soui.e  acquisition  can  be  defined  as  follows:  A  contract  for  the  purchase  of  supplies  or  services 
that  is  entered  into  or  proposed  to  be  entered  into  by  an  agency  after  soliciting  and  negotiating  with 
only  one  source.  This  definition  applies  to  new  requirements  as  well  as  an  increase  in  the  scope  and/or 
level  of  effort  of  existing  contracts. 

For  sole  source  acquisitions  over  the  aforementioned  threshold  of  $50,000  (soon  to  be  $100,000), 
the  Competition  Advjcate  Review  Board  (CARB)  meets  (on  Tuesday  of  each  week)  to  discuss  and 
approve  or  disappro.e  the  sole  source  requests.  The  original  and  twelve  copies  must  be  submitted  to 
Code  217  by  the  end  of  the  work  week  for  distribution  to  the  CARB  members  by  the  following  Monday. 

Sole  source  acquisitions  approved  by  CARB  will  be  forwarded  to  the  contracts  division.  Those 
cases  rejected  by  the  CARB  will  be  returned  to  the  originator  for  action  with  a  brief  rationale  for 
disapproval. 

Department  heads  and  project  managers  shall  ensure  attainment  of  maximum  competition  through 
the  establishment  of  initiatives  which  address  the  following  areas: 

a.  Advance  planning  of  proposed  acquisitions  to  determine  what  is  available  to  satisfy  Center 
needs,  i.e.,  to  permit  time  for  analysis  of  the  marketplace 

b.  Maximization  of  statements  reflecting  functionally  stated  performance  requirements  which 
foster  competition 

c.  Minimization  of  statements  containing  features,  designs,  or  manufacturing  processes  which 
unduly  restrict  competition 
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d.  Discounting  the  existence  of  patents  or  other  proprietary  rights  if  there  are  «..-.iiqeti':ve 
alternatives 

e.  Elimination  of  unreasonable  qualification  or  experience  requirements  which  limit  sources. 

Legal  counsel  shall  review  all  J&As  for  legal  sufficiency  and  sign;  the  cognizant  contracting  officer 
shall  certify  all  J&As;  the  cognizant  contract  negotiator/administrator  shall  sign  all  J&As.  The  cognizant 
technical  code  personnel  must  secure  the  signatures  required  for  the  “Technical  and  Requirements 
Certification;’’  the  negotiator  is  responsible  for  obtaining  all  other  approvals.  Following  the  approvals, 
the  J&A  will  be  forwarded  to  Code  217  for  review  and  then  to  Code  21  for  approval.  If  the  proposed 
action  exceeds  $100,000,  but  is  less  than  $1  million,  it  will  be  necessary  to  obtain  further  approvals 
from  Code  20  and  00.  If  the  proposed  action  exceeds  SI  million,  the  J&A  must  be  submitted  to  NAVSUP 
or  Assistant  Secretary  of  the  Navy  (Shipbuilding  and  Logistics). 

15.6.2  Brand  Name  or  F.qual 


Occasionally,  a  requirement  will  be  of  such  nature  that  it  can  be  met  by  one  of  several  commercial 
products.  When  this  situation  exists,  it  is  frequently  possible  to  make  the  acquisition  on  a  “brand  name 
or  equal”  basis.  Brand  name  does  not  require  a  full  statement  of  work,  but  does  oblige  the  requiring 
code  to  specify  all  the  technical  characteristics  which  are  necessary  to  fulfill  the  requirement.  These 
characteristics  become  the  specification  against  which  “equal”  products  can  be  measured. 

Because  of  the  increased  emphasis  on  competition,  we  have  seen  a  growing  number  of  purchase 
requests  (PRs)  which  cite  “brand  name  or  equal”  requirements.  In  “brand  name  or  equal”  solicitations, 
the  overriding  consideration  in  determining  the  equality  of  an  offered  product  is  whether  it  can  function 
to  produce  the  desired  results.  Citing  “brand  name  or  equal”  is  intended  to  be  descriptive  but  not 
restrictive,  and  is  meant  to  indicate  the  quality  and  characteristics  of  products  that  will  be  satisfactory . 
Care  must  be  taken  to  protect  the  government  from  protest  from  other  vendors  for  restrictive 
specifications;  therefore,  the  NOSC  policy  concerning  hardware,  ADPE  or  similar  acquisitions  will 
be  as  follows: 

Brand  Name  or  Equal:  If  the  responses  to  the  RFQ/RFP  result  in  only  the  specified  brand  name 
being  offered  or  the  alternates  being  rejected  as  unacceptable,  the  acquisition  will  be  considered 
a  de  facto  sole  source  requirement.  The  requirement  will  be  returned  to  the  originator  for 
resubmission  with  an  approved  sole  source 

Salient  Characteristics:  If  the  response  to  the  RFQ  RFP  reveals  that  the  salient  characteristics 
are  restrictive,  and,  in  fact,  the  characteristics  of  a  single  manufacturer,  the  acquisition  will 
be  considered  a  de  facto  sole  source  requirement  and  the  aforementioned  procedure 
implemented. 


15.7  CONTRACT  TYPES 


15.7.1  References 

For  a  full  discussion  of  the  contract  types  listed  below  see  the  Federal  Acquisition  Regulation  (FAR  i 
noted  in  the  right  column. 
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Firm  Fixed-Price 


FAR  16.2 


Other  Fixed-Price 

Fixed-Price  with  Economic  Price  Adjustment 
Fixed-Price  Incentive 
Fixed-Price  Redeterminable 
Fixed-Price  Levei-of-Effort 
Fixed-Price  with  Award  Fee 

Cost  Reimbursement 

Cost  Plus  Fixed-Fee  (CPFF) 

Other  Cost  Reimbursement 
Cost  Contract 
Cost  Sharing 
Cost  Plus  Incentive  Fee 
Cost  Plus  Award  Fee 

Indefinite  Delivery 

Indefinite  Quantity 

Requirements 

Agreements 

Cost  and  Price  Principles 
Cost  versus  Price 
Allowable  Costs 
Multiyear  Contracting 


FAR  16.203 

FAR  16.204,  FAR  16.403 
FAR  16.205 
FAR  16.207 


FAR  16  3 
FAR  16.306 


FAR  16.302 
FAR  16.303 
FAR  16.304 
FAR  16.305 


FAR  16.5 


FAR  16.7 

FAR  31 
Part  15 

FAR  17.1 


15.7.2  Service  Contracts 

The  distinguishing  characteristic  of  service  contracts  is  that  contractors  perform  identifiable  tasks 
(or  provide  advice)  rather  than  furnish  end  items  of  supply.  Services  can  require  professional  or 
nonprofessional  skills  or  combinations  of  the  two. 

The  Service  Contract  Act  (1965)  was  designed  to  ensure  that  contractors  providing  services  to  the 
government  did  not  engage  in  “wage  busting”  and  similar  practices  in  order  to  win  federal  contracts. 
The  act  requires  that,  with  certain  exceptions,  every  federal  contract  over  $2,500  for  which  the  principal 
purpose  is  to  furnish  services  must  include  a  clause  requiring  payment  to  service  employees  of  at  least 
minimum  wage  determined  to  be  applicable. 

Contracted  advisory  and  assistance  services  (CAAS)  (formerly  contractor  support  services  (CSS)) 
are  those  services  acquired  from  nongovernmental  sources  to  support  or  improve  agency  policy 
development  or  decisionmaking,  the  management  of  organizations,  or  the  operation  of  hardware  systems. 
CAAS  consists  of  the  following  four  categories: 

Individual  experts  and  consultants  (IEC) 

Studies,  analyses,  and  evaluations  (SAE) 

Management  support  sei  vices  (MSS) 

Engineering  and  technical  services  (ETS). 
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A  forthcoming  Contracts  Division  (Code  21)  instruction  will  provide  a  complete  discussion  of  this 
subject.  In  the  interim,  guidance  is  contained  in  SECNAVINST  4200. 31A,  Contracted  Advisory  and 
Assistance  Services  (CAAS). 

All  proposed  contracts  shall  be  screened  to  determine  whether  the  required  effort  is  CAAS  and 
therefore  subject  to  the  provisions  of  SECNAVINST  4200.3 1  A.  Written  justification  for  use  of  CAAS 
will  be  rigorously  reviewed  by  the  PCO  and  the  financial  manage;  authorizing  the  funds  prior  to  initiating 
the  contract  action.  CAAS  statements  of  work  shall  describe  fully  and  explicitly  the  work  to  be  performed, 
the  item(s)  to  be  delivered,  and  shall  specify  a  fixed  period  of  performance. 

SECNAVINST  4200. 27A  provides  a  full  discussion  of  experts  and  consultants,  and  personal  versus 
nonpersonal  services  contracts.  However,  the  following  definitions  apply: 

“Nonpersonal  services  contract’’  means  a  contract  under  which  the  personnel  rendering  the  services 
are  not  subject  to  the  supervision  and  control  usually  prevailing  between  the  government  and 
its  employees,  either  by  the  contract  terms  or  by  the  manner  of  its  administration. 

“Personal  services  contract’’  means  a  contract  that,  by  its  express  terms  or  as  administered,  makes 
the  contractor  personnel  appear,  in  effect,  government  employees. 

15.7.3  Indefinite  Delivery  Contracts 

There  are  three  types  of  indefinite  delivery  contracts: 

Indefinite  Quantity  —  Precise  quantities  unknown  beyond  a  specified  minimum 

Definite  Quantity  —  Quantity  known,  delivery  period  specified 

Requirements  —  Precise  needs  arid  time  period  not  initially  known. 

All  have  the  advantage  of  maintaining  government  inventory  at  minimum  levels,  while  indefinite 
quantity  and  requirements  allow  flexibility  in  ordering  as  requirements  arise.  The  titles  of  the  contracts 
generally  describe  their  uses;  funding,  however,  does  differ.  The  requirements  and  indefinite  delivery 
contracts  are  obligated  by  delivery  order  rather  than  by  the  contract  itself.  All  contracts  should  contain 
a  ceiling  (not  to  exceed  amount)  and,  in  some  instances,  a  floor. 

Federal  Acquisition  Regulation  16.504(b)  requires  that  indefinite  delivery  order  contracts  include 
a  minimum  obligation  amount  applicable  to  the  government  and  that  the  minimum  amount  be  funded 
at  the  time  of  contract  award.  The  general  view  is  that  the  minimum  should  not  be  less  than  10  percent 
cf  the  estimated  contract  ceiling. 

At  the  time  the  basic  IDTC  contract  is  to  be  initiated,  Code  21  personnel  will  instruct  the  requesting 
code  to  issue  a  7300  citing  a  job  order  set  aside  for  delivery  order  contract  minimum;  following  approval 
by  Code  122,  the  basic  contract  will  be  issued  citing  ACRN  ZZ.  To  reiterate.  Departments  will  prepare 
7300  citing  delivery  order  contract  minimum,  have  it  signed  by  approval  officials,  route  to  Code  122 
immediately,  and  absorb  into  their  overhead  any  contract  costs  incurred  as  a  result  of  the  Center  not 
placing  orders  for  the  contract  minimum. 


15.7.4  CPFF  Contracts  (Completion  versus  Term) 


A  CPFF  contract  may  take  one  of  two  basic  forms:  completion  or  term. 

The  completion  form  describes  the  scope  of  work  by  stating  a  definite  goal  or  target  and  specifying 
an  end  product.  This  form  of  contract  normally  requires  the  contractor  to  complete  and  deliver  the 
specified  end  product  within  the  estimated  cost  as  a  condition  for  payment  of  the  entire  fixed  fee. 

The  term  form  describes  the  scope  of  work  in  general  terms  and  obligates  the  contractor  to  devote 
a  specified  level  of  effort  for  a  stated  time  period. 


15.8  SOLICITATION  PHASE 

During  the  solicitation  phase  of  the  acquisition  cycle,  the  contracts  branch  has  the  primary 
responsibility  for  the  acquisition  requirements  package  (ARP).  The  requiring  code’s  involvement  during 
this  stage  is  limited.  Therefore,  a  full  discussion  of  the  subject  is  not  presented.  However,  the  contracting 
officer  may  be  consulted  if  you  have  specific  questions.  When  source  selection  and  analysis  of  contractor 
proposals  is  required,  you  will  be  contacted  by  the  contracts  branch.  Please  note  that  an  acquisition 
schedule  plan  (completed  by  the  contracts  branch)  is  prepared  for  each  ARP.  The  acquisition  schedule 
plan  establishes  acquisition  milestones  and  the  estimated  date  for  contract  award.  The  contracting  officer 
will  inform  you  if  any  deviations  in  the  schedule  are  expected  to  occur.  Once  the  source  selection  process 
is  completed,  the  contract  is  awarded. 


15.9  ADMINISTRATION  PHASE 


The  contract  administration  phase  requires  close  cooperation  among  the  team  members  to  monitor 
contractor  performance  effectively.  The  chief  areas  of  concern  which  require  effective  interface  are: 

Contract  Modifications 

Preparation  of  Delivery  Orders  &  Evaluation  of  Contractor  Proposals 

Inspection  and  Acceptance 

Government  Property 

Invoicing  and  Progress  Reports 

End  of  Contract  or  Delivery  Order  Report  Evaluation  of  Contractor  Performance. 

While  only  the  contracting  officer  is  empowered  to  change  the  terms  of  the  contract  and  to  issue 
delivery  orders,  the  requesting  code  plays  a  critical  role  in  the  contract  administration  phase.  COTRs, 
for  example,  are  required  to  scrutinize  labor  and  other  direct  costs  on  vouchers  and  to  certify  that 
such  costs  are  reasonable  and  accurately  reflect  the  work  accomplished  by  the  contractor.  Technical 
review  of  contractor  progress  reports  provides  the  most  complete  assessment  of  contract  performance 
—  only  the  requester  is  in  a  position  to  gauge  the  timeliness  and  quality  of  work  being  performed. 
The  contract  personnel  rely  upon  your  feedback  in  identifying  and  resolving  any  schedule  or  performance 
problems.  Government  property  is  another  area  in  which  contract  personnel  require  your  input.  The 
disposition  of  GFP  is  often  critical  to  the  contract  schedule.  If  the  Government  fails  to  provide  GFP, 
contract  performance  may  be  delayed.  The  inspection  and  acceptance  terms  are  set  forth  in  the  contract. 
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However,  if  inspection  and  acceptance  is  to  be  performed  at  destination,  the  requester  ML  S  I  either 
accept  or  reject  the  deliverable  in  many  cases  within  15  calendar  days,  or  acceptance  may  be  implied. 
It  is  a  very  important  function  which  cannot  be  delegated  and  which  must  be  performed  in  a  timely 
manner.  Regulations  require  an  end-of-contract  report  to  achieve  administrative  close-out.  Keep  in 
mind  that  as  the  requester,  your  responsibilities  do  not  end  with  delivery  of  the  final  product  or  service. 
Evaluation  of  contractor  performance  and  assessment  of  how  the  supply  or  service  will  benefit  your 
program  need  to  be  stated  in  the  end  of  contract  report  sent  to  you  by  the  contracting  officer. 


15.10  SPECIAL  CONSIDERATIONS 


15.10.1  ADP  Acquisitions 


Beginning  in  1965  with  the  Brooks  Act.  which  stripped  contracting  authority  for  ADP/IRM  from 
Federal  agencies  and  gave  the  authority  to  GSA,  Congress  changed  the  way  in  which  agencies  acquire 
ADP  resources.  Because  Congress  has  decided  that  we  need  to  control  ADP  purchases,  separate  and 
additional  approvals  are  required  to  procure  any  ADP-related  item  which  increases  computing  capability. 
At  present,  all  approval  memos  must  be  signed  by  Code  9103,  the  NOSC  ADP  Focal  Point  Office. 

A  standard  operating  procedure  (SOP)  is  being  developed  by  the  Contracts  Division  which  interprets 
current  policy  and  legislation  regarding  the  acquisition  of  ADPE.  Until  the  SOP  is  issued,  specific 
guidance  is  available  from  either  Code  213  or  Code  9103.  The  following  general  guidance  is  ,'rf:rcd 
(please  bear  in  mind  that  the  following  is  not  comprehensive,  and  is  subject  to  change): 

The  following  thresholds  and  approval  levels  presently  apply: 

a.  An  order  in  excess  of  $50,000  (purchase  value  for  ADPE,  annual  charge  for  software  or 
maintenance)  must  be  synopsized  in  the  Commerce  Business  Daily  (CBD);  items  available  under 
GSA  ADP  multiple  schedules  are  considered  competitive. 

b.  The  Naval  Ocean  Systems  Center  has  ADP  procurement  authority  up  to  $1  million;  everything 
over  $1  million  goes  to  the  Naval  Supply  Center,  San  Diego. 

c.  Acquisition  is  locally  approved  if  purchase  price  of  items  cosered  by  GSA  schedule  does  not 
exceed  $300,000  (usual  limit)  or  the  contract  maximum  order  limit  (MOL). 

d.  For  software  and  maintenance,  the  order  may  not  exceed  the  contract  MOL. 

e.  Code  9103  has  local  authority  to  approve  competitive  acquisitions  up  to  $10  million;  however, 
procurements  over  $300,000  require  a  delegation  of  procurement  authority  (DPA)  from  GSA. 

f.  Sole  source  over  $50,000  must  go  to  SECNAV  for  approval. 

The  primary  instructions  influencing  NOSC  ADPE  acquisition  procedures  are  the  following: 

SECNAVINST  5231. IB  of  08  March  1985 

SECNA VINST  5236.  IB  of  15  October  1980 

SECNAV  INST  5236. 2A  of  07  July  1980 

SECNAVINST  5237.1  of  07  July  1975 

SECNAVINST  5238.  IB  of  12  June  1980 

NAVSUPINST  4284.3  of  December  1985  —  ADP  Desk  Guide. 
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IS.  10.2  Small  Purchase  Procedures 


15.10.2.1  Sole  Source.  As  distinguished  from  large  purchase  procedures,  the  sole  source  justification 
(SSJ)  form  (NOSC-SD  4200/5  contained  in  NOSCINST  4200.6A)  is  to  be  prepared  for  purchase  requests 
from  over  $1,000  to  $25,000.  The  stub,  SSJ,  and  attendant  material  with  appropriate  division  head 
signature  should  be  submitted  to  the  Small  Purchase  Branch  (Code  211)  which  will  route  the  purchase 
request  and  SSJ  for  supply  approvals. 

15.10.2.2.  Purchase  Descriptions.  Purchase  descriptions  shall  not  be  written  so  as  to  specify  a  product 
or  a  particular  feature  of  a  product  peculiar  to  one  manufacturer,  thereby  precluding  consideration 
of  a  product  manufactured  by  another  company.  See  “Brand  Name  or  Equal”  for  a  discussion  of 
this  issue.  An  adequate  purchase  description  should  set  forth  the  essential  physical  and  functional 
characteristics  of  the  materials  or  services  required.  As  many  of  the  following  characteristics  as  are 
necessary  to  express  the  government’s  minimum  requirements  should  be  used  in  preparing  purchase 
descriptions: 

a.  Common  nomenclature 

b.  Kind  of  material  (type,  grade,  alternatives,  etc.) 

c.  Electrical  data,  if  any 

d.  Dimensions,  size,  or  capacity 

e.  Principles  of  operation 

f.  Restrictive  environmental  conditions 

g.  Intended  use,  including  location  within  an  assembly  and  essential  operating  condition 

h.  Equipment  with  which  the  item  is  to  be  used 

i.  Other  pertinent  information  that  further  describes  the  item,  material,  or  service  required 

j.  Purchase  descriptions  of  services  should  outline  to  the  greatest  degree  practicable  the  specific 
services  the  contractor  is  expected  to  perform. 

More  information  concerning  specifications  and  standards  is  available  in  subsection  15.5. 

15.10.2.3  Buy  American  Act.  The  Buy  American  Act  requires  that  only  domestic  end  products  may 
be  acquired  for  public  use,  except  articles,  materials,  and  supplies 

a.  For  use  outside  the  United  States 

b.  For  which  the  cost  would  be  unreasonable 

c.  For  which  the  agency  head  determines  that  domestic  preference  would  be  inconsistent  with 
the  public  interest 

d.  That  one  or  more  agencies  have  determined  are  not  mined,  produced,  or  manufactured  in 
the  U.S.  in  sufficient  and  reasonably  available  commercial  quantities,  of  a  satisfactory  quality, 
or 

e.  Purchased  specifically  for  commissary  resale. 
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Presently,  we  can  acquire  items  from  the  following  countries  without  using  die  exception  to  me 
Buy  American  Act  Form: 


Germany 

Italy 

United  Kingdom  of  Great  Britain 

Northern  Ireland 

Norway 

Belgium 

Netherlands 

Portugal 

Canada 

Denmark 

France 

T  urkev 

Luxembourg 

Spain 

I  he  exception  to  the  Buy  American  Act  Form  requires  a  signature  from  the  requesting  code,  and 
Codes  21 1  and  21 . 

15.10.2.4  Suggested  Sources.  Suggested  sources  should  be  attached  to  your  purchase  request. 
However,  everything  under  510,000  shall  be  set  aside  for  small  business  as  prescribed  by  law.  In  addition, 
sources  are  often  predetermined  by  mandatory  GSA  schedules. 

15.10.2.5  Technical  Evaluation.  At  the  request  of  211,  you  may  be  required  to  prepare  technical 
evaluations  from  vendors  responding  to  the  request  for  quotations  (RFQ).  No  specific  tormat  tor 
furnishing  your  criteria  is  required;  a  memorandum  i>  sufficient.  Keep  in  mind  that  in  order  foi  a 
contractor  to  be  considered  for  an  award,  the  product  must  meet  or  exceed  your  specifications.  Contact 
Code  211  for  further  guidance. 

15.10.2.6  Offsite  Repairs.  To  improve  repair  time  on  equipment  other  than  office  machines,  codes 
having  equipment  that  is  to  be  repaired  by  an  outside  activity  or  firm  are  requested  to  follow  the 
procedures  outlined  below. 

a.  A  stub  is  filled  out  stating  the  equipment  nomenclature,  serial  number,  quantity,  original  cost 
of  the  equipment,  and  repair  required. 

b.  The  equipment  is  turned  into  the  Traffic  Branch,  Code  223. 

c.  A  shipment  request  number  is  received  from  the  Traffic  Branch  and  written  on  the  stub. 

d.  The  stub  is  “walked  through”  the  Accounting  Division  (Code  122),  Customer  Services  (Code 
224),  and  turned  in  at  the  Purchase  Branch  (Code  211). 

15.10.2.7  Onsite  Repairs.  Each  tune  a  vendor  is  icquired  to  conic  on  board  NOSC,  a  new  stub  must 
be  issued  (except  tor  contracted  annual  maintenance).  This  is  due,  in  part,  because  labor  is  normally 
not  covered  on  a  warranty  and  authorizing  repeat  “calls”  constitutes  an  unauthorized  commitment 
it  not  covered  by  a  stub  requisition. 

15.10.2.8  Stub  Status.  To  find  out  the  status  of  your  stub  requisition  vou  can  call  \2819  or  \7160, 
or,  with  the  assistance  of  the  Consultant  ot  the  Day,  you  can  access  the  KI.V1S  mlormation  system. 
The  individual  buyer  assigned  to  your  purchase  order  should  be  contacted  if  there  is  a  change  in 
specifications  and  the  like.  However,  once  an  order  number  and  delivery  date  have  been  assigned,  contact 
Receipt  Control  tor  follow-up.  Damaged  goods  and  other  problems  with  delivery  should  be  directed 
to  Adele  Puzon,  Receipt  Control. 

Remember,  if  a  purchase  order  number  appears  on  the  RIMS  system,  it  does  not  necessarily  indicate 
that  the  order  has  been  phoned  into  the  vendor.  Many  vendors  will  not  accept  phone  orders  and  require 
a  signed  purchase  order  before  placing  the  order  which  inevitably  lengthens  the  delivery  time. 


15.10.2.9  Patents.  All  acquisitions,  regardless  of  dollar  value,  which  contain  patent  requirements 
must  be  processed  by  the  Contracts  Division. 

15.10.3  Unsolicited  Contractor  Proposals 

All  unsolicited  proposals  submitted  to  this  Center,  regardless  of  dollar  value  or  form  of  receipt, 
i.e. ,  handcarried,  personal  mail,  or  by  official  mail,  must  be  delivered  immediately  to  the  Mailroom. 
The  proposal  will  then  be  reviewed  by  the  Supply  Department  and  finally  the  cognizant  department. 
Department  heads  will  ensure  that  prompt  and  thorough  reviews  are  made  of  all  proposals  forwarded 
by  the  Supply  Department.  The  cognizant  department  shall  initiate  a  proposal  evaluation  reply  within 
30  days  and  forward  it  to  the  Supply  Department. 

Specific  guidance  concerning  this  subject  is  provided  in  NOSCINST  3900. 1C,  Procedures  for 
Handling  Unsolicited  Contractor  Proposals. 
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CONTENTS  OF  ACQUISITION  REQUIREMENTS  PACKAGE 
(Prepared  by  Tech  Code) 

Procurement  Package  Checklist  (Code  9211)  (NOSC-SD  4280/6) 

Stub  Requisition  Form  (NOSC-SD  4235/4) 

Statement  of  Work/Performance  or  Design  Specifications 
Minimum  personnel  qualifications 
Purchase  description 

Salient  characteristics  for  brand  name  or  equal 

Contract  Data  Requirements  List  (DD  1423) 

Unique  data  item  descriptions  (DIDs) 

Data  Requirements  Review  Board  (DRRB) 

DRRB  minutes  for  over  $1  million 
DMO  review  route  sheet  over  $10,000 

Delivery  Schedule,  Performance  Period,  Options 

Level  of  Effort  (LOE) 

Place  of  performance  (percentage  NOSC,  SD  facility,  DC  facility,  other) 

Labor  categories  &  estimated  hours  for  each  category 

Minimum  educational  &  experience  qualifications  for  each  category 

Detailed  Independent  Government  Cost  Estimate  includes 
Material 
Travel 
Computer 

Funding  Plan 

Type  (RDT&E,  O&MN,  OPN,  etc.) 

Accounting  data 
Availability  schedule 

Bidder’s  List  Suggestions  (Optional) 

Suggested  sources 

Proposal  Evaluation 

Source  selection  plan 

Proposal  evaluation  criteria  and  weights 

Government-Furnished  Property  (GFP)/Equipment  (GFE)  to  Contractor 
What? 

When? 

How  many? 

What  is  unit  value? 

Government-Furnished  Facility  (GFF)/Information  (GFI): 

Exact  location 
Exact  title 

Place  of  Inspection  and  Acceptance 
Notes  to  the  Negotiator 
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Presolicitation  Patent  Rights  Documentation 
Mandatory  over  $10,000 

Security  Classification  Specification  (DD  Form  254) 

If  classified  information  is  involved 

Services  Contract  Questionnaire  (NOSCINST  4200.4B) 

If  services  are  to  be  required 

Sole  Source  Justification  (SSJ)  (NOSC-SD  4200/S) 

If  noncompetitive  acquisition  is  contemplated 

Competition  Advocate  Review  Board  (CARB)  Approval 
If  sole  source 

Urgency  Statement 

If  requirement  is  urgent,  written  justification  is  required 

Conflict  of  Interest  Provision 
If  applicable 

Overtime  Justification 

If  known  in  advance  that  overtime  wages  are  contemplated  for  contractor’s  employees  under 
a  resultant  contract 

Acquisition  Plan 

Required  for  R&D  over  $2  million 

Required  for  Production  &  Services  over  $5  million 

ADPE  Approvals 

Required  when  any  item  enhancing  the  computing  capability  is  to  be  acquired  under  a  resultant 
contract 

Buy  American  Act  Exemption 
When  applicable 
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Instructions  For  Its  Use 
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INSTRUCTIONS  FOR  PREPARING  A  STUB  REQUISITION 
(NOSC-SD  4235/4-REV  11-83) 


BLOCK 

LEGEND 

INFORMATION  TO  BE  INSERTED 

1 

Code 

Code  submitting  the  requisition 

1 

Stub  Number 

Block  of  numbers  assigned  by  data  processing 

2 

Estimated  Cost 

Total  money  value  of  all  the  items  on  the  requisition 

3 

From:  Requestor’s  Name 

Name  of  person  requesting  material 

4 

Extension 

Phone  number  of  requester 

5 

Other  than  NOSC 

For  tenants:  NPRDC,  NHRC,  and  Public  Works 

6-9 

Job  order 

Assign  accounting  data  applicable  to  your  code 

10 

GFE/GFM  authorized 

Government-furnished  equipment/material  to  be 
provided 

11 

Sole  Source 

Sole  source  acquisition  documentation  provided  IAW 
NOSCINST  4200.6A 

12 

Accept  Substitute 

Choose  NO  or  YES 

13 

Date  Material  Required 

Enter  MO/DA Y/YEAR  material  is  required 

14 

Priority 

Enter  appropriate  priority  designator 

15 

Deliver  to 

Enter  name  of  person  to  whom  material  is  to  be 
delivered 

16-20 

Extension/Code/Location 

Enter  phone  extension.  Code  and  location  where 
material  is  to  be  DELIVERED 

21-22 

Requester  Signature,  Date 

Sign  and  date 

24-23 

Approval  Signature,  Date 

Authorizing  signature  and  date 

NOTE:  MUST  NOT  BE  SAME  INDIVIDUAL  CITED 
IN  BLOCK  3  OR  21 

25-27 

Internal  Approval  Signature, 
Code  and  Date 

Signature  of  individual  authorized  to  approve 
acquisition  of  certain  commodities  requiring  internal 
approval  (see  Appendix  15C)  —  must  be  signed  prior 
to  submitting  stub  to  Supply 

28 

Appropriation 

TENANT  ACTIVITIES:  enter  precisely  as  headings 
indicate 

29 

Description 

Enter  full  description,  unit  of  issue,  estimated  unit  price, 
quantity,  and  suggested  sources 

30-32 

NOT  APPLICABLE 

SUPPLY  ACTION 
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INTERNAL  APPROVAL  CONTROL  POINTS 
FOR  NAVAL  OCEAN  SYSTEMS  CENTER 


COMMODITIES  REQUIRING  APPROVALS 

ADP  (Automatic  Data  Processing)  hardware, 
software  and  supplies  (purchase,  lease,  maint.  or 
repair) 

Air  conditioning  equipment;  purchase,  maint., 
repair 

Audio/visual  equipment 

Automotive  equipment 

Blowers,  exhaust 

Boats,  engines  and  accessories 

Books,  subscriptions,  pamphlets 

Building  improvements 

Cameras:  copy,  still,  motion  picture 

Carpet 

Chemicals 

Communication  equipment 
Copy  machines 

Crystals  for  radio  communication 
equipment 

Diving  equipment  (civilian)  (SD) 

Diving  equipment  (civilian)  (Hawaii) 

Drafting  services/equipment 


APPROVALS  SIGNATURES  REQUIRED 

ADP  Focal  Point  Office 
Code  91037x2361 


Support  Branch 
Code  0066/x6447 

Audiovisual  Branch 
Code  962/X2041 

Support  Branch 
Code  0066/X6447 

Support  Branch 
Code  0066/X6447 

Service  Craft  Operations  Div. 

Code  317x6445 

Technical  Libraries 
Code  964/x6623 

Support  Branch 
Code  0066/X6447 

Audiovisual  Branch 
Code  962/.\2041 

Occupational  Safety  &  Health  Grp. 
Code  155/x6857 

Occupational  Safety  &  Health  Grp. 
Code  155/x6857 

Communications  &  Field  Eng.  Sect. 
Code  0068/X7655 

NPPS 

Bldg  A38/X7806 

Communications  &  Field  Eng.  Sect. 
Code  0068/x7655 

Civilian  Diving  Officer,  S.D. 

Code  321/x6628 

Civilian  Diving  Officer,  Hawaii 
Code  322/Hawaii  x359 

Design  &  Development  Div. 

Code  937x2214 


Drapes 

Electrical  distribution  equipment 

Electronic  test  equipment 

Energy  consuming  equipment  (heaters,  fans, 
exhaust  blowers) 

Excess  equipment  (if  required  for  NOSC) 
Excess  equipment  (leaving  NOSC) 

Facilities 

Fans 

Filing  equipment  (anything  that  holds  paper) 

Fire  extinguishers 

Forms,  printing  of 

Furniture,  office 

Furniture,  rehab 

Gases,  (all) 

Graphic  arts;  vugraphs,  slides 

Hazardous  materials  and  containers  (chemicals, 
poisons,  nuclear) 

Heaters 

Locks 

Material  handling  equipment;  dollies,  hand 
trucks,  carts,  etc. 

Nuclear  materials 


Occupational  Safety  &  Health  Grp. 

Code  155/x6857 

Support  Branch 
Code  0066/x6447 

Test  Equipment  Calibration  Grp. 

Code  953/x7234 

Support  Branch 
Code  0066/X6447 

Management  Resources  Grp. 

Code  132/X2017 

Storage  &  Disposal  Branch 
Code  221/X6573 

Support  Branch 
Code  0066/x6447 

Support  Branch 
Code  0066/X6447 

Management  Resources  Grp. 

Code  132/x2017 

Occupational  Safety  &  Health  Grp. 

Code  155/x6857 

Directives  &  Forms  Grp. 

Code  132/X5972 

Management  Resources  Grp. 

Code  132/x2017 

Storage  &  Disposal  Branch 
Code  221/x6573 

Occupational  Safety  &  Health  Grp. 

Code  155/x6857 

Audiovisual  Branch 
Code  962/x7817 

Occupational  Safety  &  Health  Grp. 

Code  155/x6857 

Support  Branch 
Code  0066/X6447 

Physical  Security  &  Loss  Prevention  Group 
Code  154/X6228 

Customer  Services  &  Material  Div. 

Code  22/x25 1 1 

Occupational  Safety  &  Health  Grp. 

Code  1557x6857 


Padlocks 

Photographic  equipment 
Plaques 

Printing  &  reproduction  (photographic) 
Projection  equipment  (video  and  photographic) 
Publications 

Radiac  equipment  (handles  radioactive  materials) 
Radio  equipment 
Sales,  office 

Safety  equipment 
Safety  glasses 

Saws,  power  (bench  or  hand  held) 

Security  containers  (safes,  file  cabinets) 

Shredders 

Signs 

Sonobuoys 
Subscriptions 
Telephone  equipment 


Physical  Security  &  Los->  Prevention  Group 
Code  154/x6228 

Audiovisual  Branch 
Code  962/x2041 

Commanding  Officer  or  Comptroller.  (Stub  will 
state  “The  expenditure  of  appropriated  funds  for 
this  purchase  is  authorized  under  applicable  laws 
and  regulations”). 

Audiovisual 
Code  962/x2041 

Audiovisual 
Code  962/X2041 

Technical  Libraries 
Code  964/x6623 

Occupational  Safety  &  Health  Grp. 

Code  155/x6857 

Communications  &  Field  Eng.  Sect. 

Code  0068/x7655 

Management  Resources  Group 
Code  112/X2017  and 

Physical  Security  &  Loss  Prevention  Group 
Code  154/x6228 

Occupational  Safety  &  Health  Grp. 

Code  155/X6857 

Occupational  Safety  &  Health  Grp. 

Code  155/x6857 

Occupational  Safety  &  Health  Grp. 

Code  155/X6857 

Physical  Security  &  Loss  Prevention  Group 
Code  154/X6228 

Information  Security  Group 
Code  152/X6842 

Support  Branch 
Code  0066/x6447 

Air  Systems  Program 
Code  021/X6377 

Technical  Libraries 
Code  964/'x6623 

Telecommunications  Group 
Code  00627x6616 


Teletype  equipment  ( when  used  yj/ADP)  ADP  Focal  Point  Office 

Code  9103/x2361 

Trade-in  equipment  (plant  accounted)  Property  <&  Stub  Control 

Code  12223/x7025 

Trailers,  (purchase  or  rental)  Management  Resources  Branch 

Code  112/x2017 

Trailers,  (repair)  Support  Branch 

Code  0066/x6447 

Training  courses  Employee  Development  Office 

Code  142/x7186 

Transportation  equipment  Support  Branch 

Code  0066/X6447 

Video  equipment  Audiovisual  Branch 

Code  962/x2041 

Word  processing  equipment  Technical  Information  Division 

Code  96/X6418 

NOTE:  Office  machines  and  typewriters  no  longer  require  internal  approvals. 


DOCUMENTATION  REQUIRING 
APPROVALS 

DD  Form  254  Contract  Security  Classification 
Specification  (Needed  when  contractor  will  have 
access  to  classified  materials) 

DD  Form  1423  Contract  Data  Requirements  List 
(When  data  will  be  a  deliverable) 

Patent  documentation  (if  patent  clauses  arc 
involved.  Code  21 1  does  not  handle;  stub  goes  to 
Contracts) 

Sole  Source  Justifications: 

$25K  and  under  (signed  by  requestor  and  division 
head) 

Over  $25K  to  $50K  (signed  by  requestor  and 
division  head) 

S50K  and  above  (signed  by  requestor,  division 
head,  department  head  and  SES  level  official) 


APPROVAL/SIGNATURES  REQUIRED 

Information  Security  Group 
Code  1527x6842 


Data  Management  Office 
Code  9203/X5817 

Patent  Office 
Code  2917x6121 


Policy  &  Planning  Branch 
Code  21 7/x7 141 
Code  211/x2819 

Contracts  Branch  Head  and  Level  Higher,  Policy 
&  Planning  Branch 
Code  217/x7141 

Competition  Review  Board  and  Policy  &  Planning 
Branch 

Code  2177x7141 


.  Vi 
V.V 
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Justification  and  Approval  (J&A)  (for  other  than 
full  and  open  competition) 

Acquisition  Plan 
Contract  Plan 

Business  Clearance  Memorandum  (BCM) 


Refer  to  Procurement  Memorandum  No.  13-85, 
dated  31  July  1985  and  Change  1  of  31  October 
1985 

[Available  from  Code  21 7/x7 141  ] 

Refer  to  Part  7  of  FAR,  DFAR  &  NARSUP 
Refer  to  Contract  Plan  form 
Refer  to  NARSUP  Part  1.690 
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SECTION  16 

BUDGET  AND  FINANCIAL  MANAGEMENT 
S.  Ryland,  Code  1211 

16.1  INTRODUCTION 


16.1.1  References 

NOSCINST  7300.5  of  27  August  1982 

NOSCINST  7300  of  5  October  1981 

NOSCINST  7300.3A  of  25  June  1982 

NOSCINST  7300.2A  CH-1  of  10  December  1982 

16.1.2  Outline 

Introduction 

References 

Outline 

Summary 

Goals  and  Objectives 
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Multiple-Funded  Customer  Orders 
Overhead/Service  Center  Number  Structure 
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Commitment/Obligation/Cost 

Overruns 


16.1.3  Summary 


See  below. 


16.2  GOALS  AND  OBJECTIVES 

Figure  16.1  introduces  the  Programs  and  Budget  Branch,  Code  121,  and  the  following  goals  and 
objectives  are  directed  toward  improving  Center  financial  management  for  the  benefit  of  both  staff 
and  program  managers. 

a.  Provide  training  and  background  necessary  to  carry  out  Financial  management  responsibilities. 

b.  Improve  communications  with  personnel  in  the  financial  arena. 

c.  Improve  Center  financial  planning  and  budgeting. 

d.  Assist  managers/staff  in  the  understanding  and  use  of  Center  financial  reports. 

e.  Explain  the  statutes  and  regulations  that  restrict  what  managers  can  do  financially. 

f.  Improve  Center  financial  management. 


16.3  THE  NAVY  INDUSTRIAL  FUND  (N1F):  WHAT  IS  IT? 


a.  The  Navy  industrial  fund  (NIF)  is  an  accounting  system  based  on  the  1949  National  Security 
Act  (see  Figure  16.2). 

b.  Revolving  fund  concept 

c.  Advantages 

Modern  accounting  methods 

Start  work  before  funds  are  available 

Simple  compared  to  other  government  accounting  systems 

All  costs  originally  charged  to  working  capital 

Cost  data  available  by  job  orders  and  cost  center 

Gives  total  cost  to  project 


Funds  Administration 
—  Record  keeping 
—  Acceptance 
—  Processing 
—  Status  report 

Budget  Staff  (Analysts) 

—  Technical  departments 
(interface  with  codes) 


•  LPS  (Laboratory  Program  Summary) 

•  A- 11  NIF  Budget 

•  Periodic/Special  Reports 

•  Budget  Staff  (for  Center) 

—  Staff  and  support  codes 
(supports  overhead  codes) 


Figure  16.1.  The  Programs  and  Budget  Branch,  Code  121. 


Figure  16.2.  The  cycle  of  operations  under  Navy  industrial  fund  (N1F>  financing. 


16.4  STABILIZED  RATES 


a.  History 

b.  Make  up  of  rate 

c.  How  it  is  applied  and  to  whom 

16.5  OVERHEAD/RATES/SURCHARGES 

16.5.1  Funded  Rates  (NOSC  Costs) 

a.  Acceleration  —  Rate  added  to  labor  to  recover  cost  of  fringe  benefits  and  leave. 

b.  General  Overhead  —  Rate  added  to  all  direct  labor  hours  to  pay  for  those  functions  which 
are  general  in  nature,  i.e.,  personnel,  supply,  comptroller,  public  works,  etc. 

c.  Productive  —  Costs  within  direct  cost  center  which  cannot  be  related  to  a  project.  Sometimes 
called  indirect  overhead. 

16.5.2  Unfunded  Rates  (Navy  Costs) 

a.  General  —  Statistical  cost  of  military  in  general  cost  centers. 

b.  Production  —  Statistical  cost  of  military  in  direct  cost  center  not  working  on  project. 

16.5.3  Surcharges 

a.  Civilian  retirement 

b.  Interest  on  investment 

c.  Administrative  surcharge 

d.  Packing 

e.  Transportation 

16.6  PROJECT  MANAGEMENT  REQUIREMENTS 

The  following  are  basic  requirements  in  the  financial  management  of  projects: 

a.  Determination  should  be  made,  with  assistance  from  the  budget  analysts,  that  the  funds  provided 
are  appropriate  for  the  work  to  be  performed. 

b.  Work  cannot  start  until  the  customer  order  is  issued. 


c.  Only  authorized  work  and  associated  costs  can  be  charged  to  job  orders  under  a  customer 
order.  All  personnel  should  ensure  that  they  use  the  correct  job  order  when  incurring  costs. 

d.  Prompt  action  should  be  taken  by  the  project  manager  to  obtain  increased  funding  when  it 
is  evident  that  work  cannot  be  completed  within  the  current  authorization. 

e.  Immediate  action  should  be  taken  to  stop  work  on  a  customer  order  when  funds  are  depleted. 
16.7  ACCEPTANCE  OF  FUNDS 


16.7.1  Funds  Required  Prior  to  Start  of  Work 

a.  Work  cannot  be  initiated  nor  costs  incurred  prior  to  receipt  and  acceptance  of  a  valid  sponsor 
order. 

b.  Exceptions:  Commander’s  Orders/Letter  of  Intent.  These  provide  the  only  means  of  starting 
work  in  advance  of  a  regular  sponsor  order. 

c.  Procedures  at  start  of  fiscal  year. 

16.7.1.1  Commander’s  Order.  The  Commander’s  Order  is  characterized  by  the  following. 

a.  It  is  limited  to  an  emergency  situation  such  as: 

Loss  or  damage  when  immediate  action  is  necessary  to  prevent  additional  loss  or  damage. 
Problem  in  Fleet  requiring  immediate  attention  to  avoid  loss  of  life  or  damage. 
Events  occasioned  by  unforeseen  security  situations. 

b.  The  Center  must  have  documented  communication  from  sponsor  that  a  funded  order  will 
be  issued  promptly. 

c.  A  Commander’s  Order  expires  within  30  days  of  issuance. 

d.  A  Commander’s  Order  is  limited  to  $250,000. 

e.  It  cannot  be  used  to  overcome  administrative  lead  time,  which  should  be  considered  in  advance 
planning. 

16.7.1.2  Letter  of  Intent.  The  Letter  of  Intent  is  characterized  by  the  following. 

a.  It  may  be  issued  by  sponsor  in  the  interest  of  economical  operations  in  advance  of  a  regular 
order. 

b.  Documentation  from  the  sponsor  is  required. 

c.  Accounting  citation  is  required. 

d.  It  constitutes  an  obligation  on  the  part  of  issuer. 

e.  Amount  of  funding  authorized  should  be  stated. 

f.  It  is  limited  to  30  days  performance  period. 

g.  NOSCINST  7300.5  of  27  August  1982. 


16.7.2  Types  of  Funding  Documents  (Reference  NOSCINST  7300  of  5  October  1981) 

Various  order  forms  are  used  by  the  Navy  and  other  agencies.  Regardless  of  the  form  used,  the 
following  are  basic  requirements: 

a.  Work  or  services  must  be  adequately  described. 

b.  The  completion  date  must  be  specified. 

c.  The  Center  must  be  substantially  in  a  position  to  perform  the  work  ordered  expeditiously. 

d.  Government-furnished  material  (GFM)  must  be  identified. 

e.  The  citation  of  funds  must  be  sufficient  to  cover  total  cost  of  the  requested  work. 

f.  Complete  accounting  data  must  be  included  for  billing  purposes. 

There  are  several  types  of  orders: 

a.  Reimbursable  Orders  —  Costs  are  initially  charged  to  the  Center  NIF  account  and  then  billed 
to  the  sponsor  for  reimbursement.  This  is  the  most  common  method  of  funding.  Under  this 
type  of  order  at  least  70  percent  of  the  work  must  be  performed  in-house. 

b.  Direct  Fund  Citations  —  These  are  issued  within  DoD.  They  are  used  when  the  request  involves 
primarily  procurement  or  travel.  The  work  is  not  financed  by  the  Center’s  NIF  account.  The 
accounting  cited  on  the  order  is  used  directly  on  any  contract  or  travel  order  issued  by  the 
Center.  The  issuer  receives  copies  of  contracts  or  travel  orders  issued  and  accounts  for  all 
obligations  and  expenditures. 

c.  Cash  Deposits  —  Required  when  work  is  performed  for  non-DoD  federal  agencies,  private 
parties,  and  state  or  local  governments.  Deposit  required  in  advance  before  work  can  start. 

16.7.2.1  Reimbursable  Orders.  This  subsection  describes  reimbursable  orders. 

a.  Work  Request  (NAVCOMPT  Form  2275) 

Issued  between  Navy  headquarters  and  field  activities,  and  between  field  activities. 
Required  for  reimbursable  work  funded  by  RDT&E,  Navy  appropriation. 

Used  for  services  of  a  continuing  nature  and  for  purposes  not  applicable  to  a  project  order. 
At  least  70  percent  of  the  work  must  be  performed  in-house. 

Completion  date  cannot  extend  beyond  expiration  date  of  financing  appropriation  or  parent 
order. 

Expiration  dates  for  RDT&E,  Navy-funded  orders  are  subject  to  incremental  funding 
restrictions  (NOSCINST  7300. 3A). 

b.  Requisition  (DD  Form  1149) 

Issued  by  Fleet  units  and  ships  to  field  activities. 

At  least  70  percent  of  the  work  must  be  performed  in-house. 

Similar  to  a  work  request  (NAVCOMPT  Form  2275). 

Can  be  accepted  as  a  reimbursable  order  or  on  a  direct  citation  basis. 


c.  Project  Order  (NAVCOMPT  Form  2275) 

Issued  between  Navy  headquarters  and  field  activities  and  between  field  activities. 
Analogous  to  contracts  placed  with  commercial  firms. 

Description  of  work  and  terms  of  order  must  be  specific,  certain,  and  definite. 

Work  must  commence  within  a  reasonable  period  of  time.  Project  order  cannot  be  issued 
if  start  of  work  is  contingent  upon  issuance  of  other  documents  or  other  authorizing  action. 

Must  serve  a  bona  fide  need  existing  in  fiscal  year  in  which  issued. 

At  least  70  percent  of  the  work  must  be  performed  in-house. 

Completion  date  can  extend  beyond  expiration  date  of  financing  appropriation.  All  work, 
including  contract  or  material  deliveries,  must  be  completed  by  date  shown  on  order. 

Cannot  be  issued  for  primary  purpose  of  extending  appropriation. 

Must  be  fully  financed  from  current  obligational  authority. 

Changes  in  scope  or  value  may  be  made  at  any  time  during  period  that  financing 
appropriation  is  available  for  obligation. 

Amendments  after  expiration  of  the  financing  appropriation  can  be  made  for: 

Price  increases  (no  change  in  scope) 

No  cost  changes  in  scope  or  completion  date. 

Project  orders  cannot  be  issued  when  the  primary  purpose  of  the  order  is: 

Major  new  construction  of  real  property 

Education,  training,  transportation,  travel,  printing,  communication,  etc. 

d.  DARPA  Order 

Issued  by  DARPA  to  other  agencies. 

At  least  70  percent  of  the  work  must  be  performed  in-house. 

Can  be  accepted  as  a  reimbursable  order  or  on  a  direct  citation  basis. 

Since  funding  is  R&D  expiration  dates  are  controlled  by  incremental  funding  policy. 

e.  Military  Interdepartmental  Purchase  Request  (MIPR)  (DD  Form  448) 

Issued  between  different  defense  agencies  —  Navy,  Army,  Air  Force,  etc. 

At  least  70  percent  of  the  work  must  be  performed  in-house. 

Can  be  accepted  as  a  project  order  if  it  meets  qualifications. 

Completion  date  requirements  vary  depending  on  appropriation. 

f.  Orders  from  non-DoD  federal  agencies 

Issued  by  other  federal  agencies  such  as  NASA,  DOE,  NOAA,  Interioi,  etc. 

Forms  vary  from  agency  unique  purchase  orders  to  memorandum  of  understanding. 
At  least  70  percent  of  the  work  must  be  performed  in-house. 

Completion  dates  vary  depending  on  agency  rules. 

Must  have  cash  advance. 
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16.7.2.2  Direct  Cite  Orders.  Documents  which  are  to  be  direct  cited  such  as  requests  for  contractual 
procurement  (RCPs)  or  “direct  cite”  military  interdepartment  purchase  requests  (MIPRs)  are  accepted 
through  a  slightly  different  procedure  than  reimbursable  orders: 

An  LPS/DD1498  is  not  required. 

If  the  requested  procurement  is  not  mission  related,  approval  from  CNM  is  required  before 
the  Center  accepts. 

RCPs  are  reviewed  by  the  Supply  Officer,  Code  20,  prior  to  acceptance. 

NOSC  job  orders  are  not  used  when  initiating  contracts.  The  accounting  cited  on  the 
RCP  or  M1PR  is  used  directly  on  the  contract.  A  copy  of  the  RCP  or  MIPR  should  be 
attached  to  each  stub  issued. 

This  subsection  describes  direct  fund  citations. 

a.  Request  for  Contractual  Procurement  (RCP)  (NAVCOMPT  Form  2276) 

Issued  between  Navy  headquarters  and  field  activities  and  between  field  activities. 

Must  relate  to  in-house  NOSC  program;  otherwise  CNM  approval  for  acceptance  is 
required. 

Should  be  used  for  specific  contracts. 

Should  not  be  used  for  smaller  material  purchases. 

If  supporting  in-house  services  are  required,  these  should  be  funded  by  a  separate 
reimbursable  order. 

Expiration  dates  vary  with  appropriation. 

b.  Requisition  (DD  Form  1149) 

Issued  by  Fleet  units  and  ships  to  field  activities. 

Same  rules  as  for  RCP. 

c.  Letter  of  Authorization  for  Travel 

Issued  between  Navy  activities  to  fund  specific  travel  requirements. 

d.  DARPA  Order 

Issued  by  DARPA  to  other  agencies. 

Same  rules  as  for  RCP. 

e.  Military  Interdepartmental  Purchase  Request  (MIPR)  (DD  Form  448) 

Issued  between  different  defense  agencies  —  Navy,  Army,  Air  Force,  etc. 

Same  rules  as  for  RCP. 

16.7.3  Center  Acceptance  Procedures 

All  funding  documents  received  on  Center  are  forwarded  to  Code  1211.  After  internal  review  and 
acceptance  by  the  project  manager,  Code  1211  accepts  the  document  and  returns  it  to  the  sponsor. 
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An  accepted  reimbursable  order  becomes  an  obligation  on  the  sponsor’s  books  Figure  16.3  presents 
the  fund  acceptance  procedures  for  reimbursable  orders. 

Before  acceptance  of  a  fund  document  the  budget  analyst  and  the  project  manager  must  review 
it  to  ensure  plans  are  current  and  that  various  requirements  relating  to  the  document  are  met. 

The  following  questions  are  to  be  considered  when  accepting  a  new  funding  document: 

Is  the  LPS  current  and  approved  by  appropriate  Center  official? 

Will  70  percent  of  the  work  on  a  reimbursable  order  be  performed  in-house?  If  not,  an  RCP  or 
other  direct  cite  document  is  required. 

Is  the  funding  sufficient  to  complete  work? 

Is  the  completion  date  adequate? 

For  RDT&E  funded  work  does  the  completion  date  conform  to  incremental  funding  requirements? 
For  project  orders:  are  special  requirements  met? 

Is  the  type  of  funding  appropriate  for  the  work  being  performed? 

Are  purchases  of  investment  items  planned?  Are  funds  appropriate? 

Are  special  reporting  or  accounting  requirements  specified  in  the  document?  Can  these  be 
accommodated?  By  whom? 

16.7.4  Comparison  of  Appropriations 

Figme  16.4  and  Table  6.1  take  a  look  at  appropriations  from  an  expense/ investment  point  of  view, 
while  Table  16.2  presents  the  types  of  funding  appropriations.  Table  16.3  presents  DoD  program  elements 
with  a  detailed  notation  of  the  research  and  development  categories. 

The  following  listings  provide  a  comparison  of  the  different  uses  of  appropriations. 

a.  Research  and  Development 

Basic  and  applied  research 

Studies  —  theoretical,  feasibility,  design,  engineering,  cost  effectiveness 

Experimental  or  prototype  hardware  —  design,  develop,  fabricate,  test 

Software  for  tactical  and  strategic  systems  (requiring  hardware  R&D)  —  design, 
development,  integrate,  test 

Product  improvement  —  when  expanding  current  performance  envelope 
Development  test  and  evaluation  (DT&E) 

Initial  operational  test  and  evaluation  (IOT&E) 

Investment  items  and  expense  necessary  for  an  R&D  project 

b.  Procurement  Appropriations  (APN,  OPN,  SCN.  WPN) 

Investment  hardware  and  software  not  requiring  R&D 

Production  direct  support  —  production  engineering,  quality  assurance,  production  testing, 
equipment  assembly  (excludes  production/ procurement  program  management) 
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Figure  16.3.  Fund  acceptance  procedure  for  reimbursable  orders. 


Expense  Investment 

RDT&E  •  • 

Operators  and  Maintenance  • 

Procurement 

Aircraft  (APN)  • 

Weapons  (WPN)  • 

Ships  (SCN)  • 

Other  (OPN)  • 

Other 
FMS 

Other  Agencies 
Special  Deposits 


Figure  16.4.  Appropriations. 
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Table  16.1.  Investment/  Lxpense 

(specific  definitions  are  contained  in  NAVCOMPT  manual  vol.  7,  paragiaph  075001) 


Investment 

Equipment  costing  S3, 000  or  more  (including 
software) 

Labor  used  in  assembly,  production,  or  construc¬ 
tion  of  investment  item  and  for  direct  support 
costs. 

Spare  assemblies  and  parts  centrally  managed. 
Modification  kits  and  assemblies. 

APA  material 

Facility  construction  and  direct  support  costs, 
design  engineering. 

Initial  outfitting  of  major  end  item  of  equipment. 

Ammunition  and  explosives 

Procurement/production  direct  support 

Production  testing 
Quality  assurance 
Product  engineering 
Equipment  assembly 


Expense 

Equipment  costing  less  than  S3, 000. 

Labor  used  for  operations,  maintenance,  and 
services. 

Spare  assemblies  and  pans  not  centrally  managed. 

Nonrepairable  spare  parts. 

Consumable  material  and  supplies. 

Movable  furnishings,  furniture,  and  galley  equip¬ 
ment  (some  exceptions). 

Services 

Rental  and  leases  (some  exceptions) 

General  motion  picture  procurement  or 
development 

Procurement/production  program  management 

Maintenance  and  repair 

Labor,  material,  and  equipment  incorporated 
into  end-item. 

Replacement  of  equipment  or  assemblies 
installed  in  major  end-item. 

Modification  (alteration,  conversion,  moder¬ 
nization) 

Labor  (except  for  ship  conversion) 

Locally  procured  material 

Minor  construction  (less  than  $200,000  not  funded 
by  MILCON  or  family  housing  appn). 


Table  16.3.  DoD  program  dements  with  a  detailed  notation  ol  the  research  and  de\ eicpmcn;  categories. 

DoD  Program  Elements 

0  Support  of  Other  Nations 

1  Strategic  Forces 

2  General  Purpose  Forces 

3  Intelligence  and  Communications 

4  Airlift  and  Sealift 

5  Guard  and  Reserve  Forces 

6  Research  and  Development 

7  Central  Supply  and  Maintenance 

8  Training,  Medical,  and  Other  General  Personnel  Activities 

9  Administration  and  Associated  Activities 

R&D  Categories 

6.1  Research .  ..  Scientific  study  and  experimentation  directed  toward 

fundamental  knowledge  and  understanding  needed 
for  the  solution  of  identified  military  problems. 

6.2  Exploratory  Development . All  effort  directed  toward  the  solution  of  specific 

military  problems,  short  of  major  development 
programs. 

6.3  Advance  Development . All  projects  which  have  moved  into  the  development 

of  hardware  for  experimental  or  operational  test 

6.4  Engineering  Development . Those  development  programs  being  engineered  for 

service  use  but  which  have  not  yet  been  approved  for 
procurement  or  operation. 

6.5  Management  and  Support . Development,  test,  and  evaluation  not  separately 

provided  for.  Includes  facility  and  military  support 
resources. 

6.6  Operational  Systems  Development  . All  effort  having  the  primary  objective  of  produci- 

bility  demonstration  and  R&D  phases  of  final  service 
test  of  logistical  and  operational  employment  of  a 
system  approved  for  procurement  and  operational 
deployment. 


Product  improvement  —  when  item  is  in  production  and  improvement  is  within  current 
performance  envelope 

Test  articles  —  acquisition  of  test  articles  required  for  follow-on  operational  test  and 
evaluation  (FOT&E) 

c.  Operations  and  Maintenance 

Fleet  support 

Maintenance  and  repair  of  operational  items 

Product  improvement  of  items  in  operational  inventory  when  no  longer  in  production 
and  when  improvement  is  within  current  performance  envelope 

Follow-on  operational  test  and  evaluation  (FOT&E)  (except  for  acquisition  of  test  articles) 

Expense  items  —  labor,  material,  supplies,  travel,  services 

Equipment  costing  less  than  $3,000 


16.7.5  Navy  RDT&E  Incremental  Funding  Policy 

a.  Specific  guidance  is  contained  in  NOSCINST  7300.3A  of  25  June  1982. 

b.  Goal  —  to  budget  and  finance  RDT&E  work  in  12-month  increments  coincident  with  the  fiscal 
year. 

c.  Effect  on  Center  Funding 

Limits  period  of  time  during  which  the  Center  can  use  funds 
Limits  amount  of  funding  that  can  be  placed  on  contracts 

d.  General  Policy  (see  Figure  16.5) 

RDT&E  appropriation  is  a  2-year  appropriation,  but  it  is  usually  limited  to  12  months 
availability  for  labs. 

The  12-month  availability  period  can  be  extended  for  award  of  contracts  for  material 
or  equipment  which  were  initiated  during  first  12-month  increment,  where  award  was 
delayed  because  of  contractual  or  technical  problems. 

Service  contracts  award  by  the  Center  with  RDT&E  funds  cannot  extend  beyond  the 
completion  date  on  sponsor  work  request. 

Multiyear  contracts  funded  by  RDT&E  should  be  funded  in  annual  increments  to  coincide 
with  fiscal  year. 

Fully  funded  short  term  contracts  (18  months  or  less)  may  be  issued  where  it  is  not  feasible 
to  increment. 

When  budgets  are  prepared  for  RDT&E  funded  projects,  managers  should  take  into 
account  these  incremental  funding  requirements. 
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9  mos 

12  mo«. 
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COMMENT 

Coincident  with  the  FY  (objrci/ve). 

Not  coincident  with  the  FY. 

All  cfTort  within  part  of  the  FY. 

Award  made  late  in  FY.  maximum  permitted  duration  a 
IS  monthx 

Award  made  in  second  year  of  availability .  maximum 
permitted  duration  u  18  month*.  Budget  cannot  be 
baud  on  due  type  of  ftmdmf  pfon 

Coincident  wuh  the  FY  (objecuve). 

Pint  increment  made  to  coincide  with  end  of  FY 

Exception  SFC^ AV  epprrwil  u  required  if  this  patterr 
a  u*d  in  budget  formulation  where  the  fir  it  increment 
extendi  beyond  the  end  of  the  rim  year  and  the  wcond 
increment  a  made  to  coincide  with  the  end  of  the 
Kcond  year.  Difficulties  in  execution  may  require  this 
patern. 

-►Exception  SECNAV  approval  u  raquiml  if  thu  pattern 
is  u*ed  in  budget  formulation,  where  the  intferrwntf 
•re  not  coincidental  with  the  fiscal  y«ar.  Difficult**  ui 
execution  may  require  lha  pattern 

Award  made  in  second  yea/  of  a *» d a ^*1  < 'v  funding  pattern 
as  ah  own  tn  example  6  restored  by  end  of  next  succeeding 
I*  .  Budget  cannot  be  baaed  on  tho  type  of  fund  mg  plan 


^Maximum  duration  of  initial  mere  mem  n  36  months  from 
date  of  award. 

Maximum  duration  of  any  increment  after  the  initial  incre¬ 
ment  i*  1 2  momhi  from  date  of  renewal. 


Institutional  funding. 


^Reimbursable  orders,  pbnned  ixic/emeot  may  extend  up 
lo  3  months  mto  the  following  FY. 

Reimbursable  order  timed  m  second  yes  of  availability, 
second  increment  funded  ui  second  >w  of  availabilhy. 
maximum  duration  ta  6  months  of  the  foOowtnj  FY. 
Budget  cannot  be  baaed  on  this  type  of  funding  plan 


Figure  16.5.  RDT&E  incremental  funding. 
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16.8  SPENDING/CONTROL  OF  FUNDS 


16.8.1  NOSC  Project  Numbering  Structure 

The  NOSC  numbering  structure  integrates  a  project  number  with  customer  order  and  job  order 
numbers.  This  provides  for  identification  of  different  areas  of  work  at  the  project  number  level,  funds 
control  at  the  customer  order  level,  and  cost  accounting  by  job  order.  A  complete  explanation  of  this 
structure  is  shown  in  Figure  16.6. 

Example:  CC54831A01 

Project  No . CC54 

Customer  Order . CC54831A 

Job  Order . CC54831A01 

16.8.1.1  Project  Number.  The  project  number  (the  first  four  alphanumeric  characters)  identifies 

the  type  of  work  to  be  performed  relative  to  various  Center  mission  areas  and  to  the  specific  project 
effort  within  that  area  of  work.  These  mission  area  indicators  are  listed  in  Table  16.4. 

16.8.1.2  Customer  Order  Number.  The  customer  order  (the  first  eight  alphanumeric  characters)  is 
an  extension  of  the  project  number,  and  it  identifies  the  NOSC  division  managing  the  customer  order, 
the  fiscal  year  of  the  funding,  and  the  specific  sponsor  order  which  is  funding  that  work.  This  is  the 
level  at  which  funds  are  allocated  and  funds  control  maintained.  In  certain  limited  cases  customer  orders 
may  be  multiple  funded  (see  subsection  16.8.2  for  the  ground  rules). 

16.8.1.3  Job  Order  Number.  The  job  order  is  an  extension  of  the  customer  order  formed  by  adding 
two  digits.  It  is  used  to  identify  specific  segments  of  cost  under  a  customer  order  as  needed  for 
management  purposes.  The  total  10-character  job  order  is  used  on  all  financial  transactions  such  as 
timecards,  travel  orders,  and  stub  requisitions  for  procurement. 


16.8.2  Multiple-Funded  Customer  Orders  (from  NOSCINST  7300.3A  of  25  June  1982) 

Mixing  of  funds  on  a  customer  order  is  permitted  in  certain  exceptional  cases: 

a.  Work  or  end  product  described  on  each  fund  document  is  the  same. 

b.  Funds  documents  are  of  the  same  type,  i.e.,  all  work  request,  project  orders,  or  other  type. 

c.  Work  described  on  the  fund  documents  is  identical  in  scope,  i.e.,  all  require  identical  portions 
of  various  cost  elements  —  labor,  material,  contracts,  etc. 

d.  The  work  schedule  starting  and  completion  dates  are  the  same. 

e.  The  fund  documents  contain  identical  terms  —  all  reimbursable  type  orders. 

f.  Fund  documents  are  received  in  same  year.  Carryover  funds  cannot  be  mixed  with  new  funds. 

g.  Funds  are  under  the  same  appropriation  —  all  RDT&E,  all  O&MN,  etc. 

h.  RDT&E  funds  are  within  the  same  R&D  category,  i.e.,  all  6.1,  6.2,  6.3,  etc. 
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NOSC  Job  Order  Example:  WT01614A01 


Project  Number 

(4  characters)  — 

WT01 

Customer  Order 

(8  characters)  — 

WT01614A 

NOSC  Job 

Order 

(10  characters)  — 

WT01614A01 

Mission 

Specific 

Cognizant 

Fiscal 

Customer  Order 

Job  Order 

Area 

Project 

Division 

Year 

Serial 

Serial 

WT 

01 

61 

4 

A 

01 

Project  Number  —  A  four-character  code  identifying  the  NOSC  project. 


Character  Number 
1 
2 

3  &  4 


Significance 

Alpha  code  for  major  mission  area 
Alpha  code  for  mission  subcategory  area 
Numeric  serial  code  identifying  specific  project 


Customer  Order  Number  —  An  eight-character  alphanumeric  code  identifying  specific  funding  or  tasks 
under  a  project.  It  includes  the  project  number  plus  four  additional  characters. 

Character  Number  Significance 

5  &  6  Two-character  numeric  code  identifying  the  cognizant  division. 

7  Single  numeric  code  identifying  the  fiscal  year  in  which  the  funds  are 
allocated  (e.g.  1984-4). 

8  Single  alpha  code  identifying  the  customer  order  assigned  funding.  Fund 
control  is  exercised  at  this  level. 


NOSC  Job  Order  Number  —  A  10-character  alphanumeric  code  that  identifies  specific  tasks  or  work 
areas  under  a  customer  order  as  needed  for  management  control.  It  includes  the  customer  order  number 
plus  two  additional  characters. 

Character  Number  Significance 

9  &  10  Two-character  numeric/alpha  serial  code  used  for  breakout  of  the  customer 

order  into  job  packages  recording  and  reporting  purposes. 


Figure  16.6.  The  10-character  NOSC  job  order  number  explained. 
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Tabic  16.4.  Mission  area  indicators  in  the  NOSC  project  numbering  system  (1  of  2). 


SYSTEMS  ANALYSIS 
AS  Systems  Analysis 

COMMAND,  CONTROL,  AND  COMMUNICATIONS 

CC,  CD  Command  Control 

CE  C3  Systems  Human  Engineering 

CF  C3  Integration  Facility 

CG,  CH  Communications  —  General 

CM,  CN  Communications  —  Naval  Vessels 

CS  C3  Systems 

CT  Tactical  Sensors  Pn 


E.  ENGINEERING  TECHNOLOGY 

EC  Computer  Sciences 

EE  Electronics  Technology 

EM  Mechanical  Engineering 

ES  Real-Time  Simulation 

ET  Manufacturing  Technology 

F.  FLEET  SUPPORT 

FA  Fleet-Funded  Programs 

FM  NAVMAT  Fleet  Support 

FN  NSAP  Programs 

H.  HYDROMECHANICS 

HM  Hydromechanics 

M.  MARINE  SCIENCE  AND  TECHNOLOGY 

MA  Environmental  Acoustics 

MB  Airborne  Acoustics 

MB  Deployed  Systems 

ME  Environmental  Quality 

MJ  Radiation  Physics 

MM  Marine  Mammal  Technology 

MP  EM/EO  Propagation 

MR  Arctic  RAD 

MS,  MT  Systems  &  Technology 

N.  EQUIPMENT  A  FACILITIES 

NC  Construction/ Installation 

NE  Equipment/Instrumentation 

NI  NICRAD  Program 


Project  Number 
4  Characters 
X  X  XX 

C Serial  number  identifying 
specific  project. 

Letter  code  identifying  sub- 
— category  of  mission  area. 

Letter  code  identifying  major 
-mission  area. 

Sample: 

NEARTIP  —  WT01 
ALWT  —  WT02 


PROPOSAL 


Proposals/Planned  Projects 
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Table  16.4.  Mission  area  indicators  in  the  Nl<  >SC  project  numbering  system  t 


Sv 


S.  SURVEILLANCE  PROGRAMS 

SA  Aerospace  Surveillance 

SS  Surface  Surveillance 

ST,  SU,  SV  Undersea  Surveillance 

SX  Ocean  Surveillance 

SY  Surveillance  Systems 

T.  TESTS  AND  SERVICES 

TA  ADP/Computer  Services 

TC  Calibration  Services 

TD  SACS/FORACS 

TE  Environmental  Testing 

TG  Graphics 

TM  Machine  Shop  Services 

TR  Range  Testing 

TT  Tenant/Military  Support 

W.  WEAPONS  SYSTEMS 

WC  Fire  Control  Programs 

WL  Launcher  Programs 

WM  Missile  Programs 

WS  Sonar  Programs 

WT  Torpedo  Programs 

WX  Countermeasure  Programs 

X.  MISCELLANEOUS  SERVICES 
XA,  XB,  XC, 

XD,  etc.  Miscellaneous 

Z.  INDEPENDENT  PROGRAMS 

ZD,  ZE  Independent  Exploratory  Development  (IED) 

ZR,  ZS,  ZT  Independent  Research  (IR) 


Where  a  fund  document  contains  multiple  accounting  line  items  (ACRNs)  for  the  same  project, 
these  ACRNs  must  be  established  as  separate  customer  orders  if  sponsor  task  statements  correlate  to 
the  separate  line  item.  Where  no  correlation  is  possible,  the  funds  can  be  mixed. 

Funding  segments  of  a  multiple-funded  customer  order  will  be  billed  on  a  percentage  basis. 


16.8.3  Overhead/Service  Center  Number  Structure 

A  10-digit  “job  order”  structure  is  used  to  account  for  overhead  and  service  center  expense.  It 
does  not  contain  a  project  number.  Instead,  it  relates  to  the  cost  center  having  cognizance  of  the  work 
and  the  type  of  expense  based  on  prescribed  NIF  expense  elements.  The  NOSC  expense  account 
numbering  structure  is  shown  in  Figure  16.7. 


NOSC  Expense  Account  Number  Example:  7140042001 


Type  of 

Cognizant 

Expense  Element 

Expense  Account 

Expense 

Code 

or  Function 

Serial 

7 

1400 

420 

01 

(General 

Overhead) 

(Personnel  Division 

Officer) 

(Travel) 

(1st  Serial) 

Figure  16.7.  The  10-digit  NOSC  expense  account  numbering  structure. 


16.8.3.1  Type  of  Expense.  A  one-digit  code  identifies  the  type  of  expense  (4-service  center,  6-indirect 
expense,  7-general  expense,  8-special  center-funded  function,  etc.). 

16.8.3.2  Organizational  Code.  A  four-digit  code  identifies  the  cognizant  organization  at  the  section 
level. 

16.8.3.3  Function.  A  three-digit  code  identifies  the  function  where  such  breakout  is  required  (for 
example,  training,  maintaining  of  building,  etc.)  or  identifies  the  element  of  expense  (for  example, 
gas,  telephone  service,  etc.).  A  representative  listing  of  the  expense  element/ function  codes  is  presented 
in  Table  16.5. 


TaMa  16.5.  The  expense  element/ function  code  structure. 


10 

Salaries 

40 

Miscellaneous 

13 

Administrative  salaries 

41 

Training 

14 

Salaries  and  wages  (other) 

42 

Travel 

20 

Material 

43 

Printing,  reproduction,  blueprinting 

23 

Operating  supplies 

45 

Telephone 

30 

Contracts 

50 

Utilities  (PW  only) 

31 

Major  contractual  services  ( >  10K) 

60 

Repair  and  maintenance 

32 

Minor  contractual  services  (<10K) 

61 

Buildings 

33 

EAM/ADP  rentals 

80 

Adjustments 

35 

Transportation 

90 

Transfers 

38 

Technical  information 

39 

Computer 
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16.8.3.4  Serial.  A  two-digit  code  which  can  be  used  as  necessary  to  categorize  expenses  tor  special 
purposes. 


16.8.4  Commitment/Obligation/Cost 


a.  Use  of  funds  and  expiration  dates 


Spending  Action  Required  by  Expiration  Date 

Work  Request  . Obiigate/performance  on  service  contract 

RCP . Obligate/ performance  on  service  contract 

Project  Order . Completion  of  work 

b.  Definitions 

Commitment . Reservation  of  funds  for  planned  procurement 

Obligation . Legally  binding  order/ contract/task 

Cost . Receipt  of  goods  and  services 


c.  Types  of  transactions 

The  types  of  transactions  are  charted  in  Figure  16.8. 


Labor  and  Overhead 
Travel 

Stub  Requisition 

Purchase  Order 

Contract 

Delivery  Order 

Work  Request/MIPR/P.O. 
(when  accepted) 


Commitment  Obligation  Cost 

X 

X 

X 

X 

X 

X 

X 


X 


X 


RCP  —  when  contract  is  awarded. 
GBL  (government  bill  of  lading) 
Invoice 


Figure  16.8.  Types  of  transactions. 


X 


16.S.5  Overruns  (Reference  NOSCINST  7300.2A  CH-1  of  10  December  1982) 


a.  Prevention  of  overruns 

Review  spending  trend. 

Determine  if  funds  are  sufficient. 

Advise  sponsor  ahead  of  time. 

Obtain  additional  funds. 

Stop  work  —  advise  all  personnel  working  on  project. 

b.  Center  write-off  authority 

NOSC  can  absorb  small  variances  (except  for  FMS  and  private  parties)  as  follows: 

Sponsor  orders  of  $10,000  or  less  —  the  smaller  of  $500  or  10  percent  of  the  order. 
Sponsor  orders  over  $10,000  —  the  smaller  of  $1,000  or  5  percent  of  the  order. 
These  variances  are  written  off  to  Center  operating  results  by  the  Budget  Office,  Code  121 . 

c.  Disposition  of  overruns 

Cancel  or  reduce  outstanding  commitments. 

Determine  if  undelivered  material  is  needed  by  another  project.  Initiate  transfer. 
Prepare  letter  to  sponsor  requesting  additional  funds. 

If  the  sponsor  officially  advises  that  funds  are  not  available,  the  overrun  is  charged  to 
Center  operating  results. 

d.  Overrun  reports 

Jeopardy 
Stop  work 
Overrun  notification 
Overruns  outstanding 
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SECTION  17 
DESIGN  REVIEW 
J.  Rasmussen,  Code  902 


17.1  INTRODUCTION 


17.1.1  References 

NOSCINST  4855.1,  NOSC  Product  Assurance  Program 
NAVMAT1NST  3000.1  A,  Reliability  of  Naval  Material 

17.1.2  Outline 

Introduction 
References 
Outline 
Summary 

Design  Review  Goals 
Product  Assurance 
Types  of  Reviews 
Mandatory  Reviews 
Negotiable  Reviews 
Reviews  Not  Required 
Responsibilities 
Department  Head 
Design  Review  Committee 
Organization  of  the  Design  Review  Committee 

17.1.3  Summary 

This  section  explains  NOSC’s  design  review  policies  and  states  Center  policy  relative  to  design 
approval  and  release  of  NOSC-developed  components,  subsystems,  systems,  and  major  items  of  system 
software. 


17.2  DESIGN  REVIEW  GOALS 

The  primary  goal  of  the  design  review  process  is  to  ascertain  that  the  development  programs  at 
the  Center  have  a  high  probability  of  success  in  meeting  their  technical  requirements  and  will  be 
operationally  effective  and  sustainable  when  delivered  to  the  Fleet.  The  process  is  provided  to  assist 
the  program  manager  in  this  regard.  In  addition,  these  independent  reviews  will  provide  the  basis  for 
advice  to  the  technical  director  on  all  technical  and  material  matters  of  concern  pertaining  to  the 
development,  operation,  and  production  of  a  component,  subsystem,  system,  or  major  item  of  system 


software,  developed  by  and  which  is  the  responsibility  of  the  Center,  for  decisions  related  to  the  fielding 
of  the  product.  The  design  review  committee  shall  also  provide  this  function  in  those  instances  where 
the  Center  acts  as  design  agent  or  technical  direction  agent  or  has  other  major  design  responsibility 
for  systems  or  subsystems.  The  committee  shall  also  provide  this  function  in  those  instances  where 
the  Center  has  other  significant  responsibilities  related  to  the  use  of  unproven  items  as,  for  example, 
in  operational  exercises  or  scientific  sea  tests.  Figure  17.1  offers  a  sample  outline  for  a  major  system 
presentation. 


17.3  PRODUCT  ASSURANCE 

Guidance  and  reference  material  related  to  the  NOSC  product  assurance  program  are  provided 
in  NOSCINST  4855.1. 


17.4  TYPES  OF  REVIEWS 

NAVMATINST  3000.1  A  states  that,  at  a  minimum,  the  design  reviews  shall  include  preliminary 
design  review  (PDR)  of  the  design  approach  prior  to  initiation  of  detailed  design;  a  critical  design  review 
(CDR)  of  the  detailed  design  prior  to  drawing  release  and  fabrication  of  formal  test  articles;  a  design 
certification  review  (DCR)  of  the  final  design  subsequent  to  qualification  testing  and  prior  to  Navy 
operational  test  and  evaluation  and  production  start;  and  a  final  production  review  (FPR)  at  the  time 
of  a  first  article  configuration  inspection  (FACI)  of  the  as-produced  design  following  manufacture 
and  acceptance  testing  of  the  first  end  item  configured  for  delivery  to  the  Fleet.  If  standard  reviews 
are  conducted  by  the  sponsor  as  parts  of  the  system  development  process,  the  design  reviews  in  concert 
with  these  scheduled  reviews.  Figure  17.2  indicates  timing  of  reviews  within  the  major  systems  acquisition 
cycle. 


17.4.1  Mandatory  Reviews 

All  major  acquisition  programs  require  mandatory  reviews.  These  include  development  of  large 
systems  with  many  components  destined  for  large  production  numbers  and  eventual  long  life  in  the 
Fleet  (e.g.,  MK  50  torpedo,  MILSTAR  SATCOM). 

All  man-machine  interfaced  programs  involving  safety  considerations  require  mandatory  reviews. 

All  hardware/software/firmware  systems  leaving  NOSC  for  installation  supporting  any  DoD 
activity,  even  for  a  limited  installation  period,  also  require  mandatory  reviews. 


17.4.2  Negotiable  Reviews 

Reviews  are  negotiable  for  minor  systems  or  feasibility  models  designed  for  limited  production. 
Minor  systems  include  smaller  systems  with  fewer  components  for  feasibility  demonstration  or  limited 
production  that  may  still  have  long  life  in  the  Fleet  (e.g.,  AUSS  (Advanced  Undersea  Search  System), 
SLC  (Submarine  Laser  Communication)  system,  etc.). 

Reviews  are  also  negotiable  for  nondeliverable  demonstration  testbeds. 


SAMPLE  MAJOR  SYSTEM  PRESENTATION  OUTLINE 


INTRODUCTION 

Purpose  of  Review 

BACKGROUND 

Operational  Requirement 
Program  Summary  WBS  (Sponsor’s) 

Narrative  System  Description 
Program  Objectives 

System  Performance 

Cost 

Schedule 

Management  Approach  (Sponsor’s) 

Program  Plan 
Acquisition  Plan 
Delegation  of  Responsibilities 

-  Sponsor 

-  Contractors 

-  Centers/Labs/etc. 

-  NOSC 

-  Tasking  Documents 

-  Interface  Agreements,  Work  Agreements 
MANAGEMENT  OVERVIEW 

Subprogram  WBS  (NOSC) 

Organization 

Accountability  Matrix  (WBS  vs  Organization  Chart) 
Assignments  of  Responsibility 
Management  Practices 
Planning 
Reporting 

Cost/Schedule  Tracking  and  Analysis 
Design  Review  Schedule 
Management  Review  Schedule 
Schedule 

Milestone  Objectives 
Budget 
Fiscal 

Current 
Out  Years 
Manpower 

Total  and  by  Departments 
Other  Resources 
Procurement  Plans/Status 
Subsystems 
Components 
Support/Services 


Figure  17.1.  Sample  major  system  presentation  outline  (I  of 
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TECHNICAL  PROGRAM 

System  Engineering 

System  Requirements  (Specifications) 

Performance 

Environmental 

Life-Cycle  Profile 
System  Level 
System  Functions 

Functional  Allocations 
Subsystem  Requirements 
Performance 
Environmental 
Reliability  Allocations 

Safety  Requirements  (Operational/Handling/Storage) 
Maintainability  Allocation  and  Philosophy 
Cost  Allocation 

Test  &  Evaluation 
System  Level 
Subsystem  Level 
Performance 

Environmental 
Reliability 
Maintainability 
Safety  (Systems) 

Human  Factors 
Documentation  Plans/Status 
Level 

Verification/Validation 
Product  Assurance  Plans/Status 
Quality  Control 
Producibility 

Configuration  Management 
ILS  Plans/Status 
Support  Concept 
Responsibilities 
Manuals 

Support  Equipment 

CONCLUSIONS 

Risks 

Technical 

Cost 

Schedule 

Problem  Summary 
Problems 
Solutions/Effort 
Recommendations 


Figure  17.1.  Sample  major  system  presentation  outline  (2  of  2) 


Figure  17.2.  Major  defense  systems  acquisition  cycle 


17.4.3  Reviews  Not  Required 

Test  instrumentation 
Nondeliverable  test  hardware 
Nondeliverable  experimental  devices 
Minor  support  equipment 


17.5  RESPONSIBILITIES 

17.5.1  Department  Head 

It  is  the  responsibility  of  every  department  head  to  identify  in  a  timely  way  those  components, 

subsystems,  systems,  and  major  items  of  software  which  must  undergo  design  review. 

17.5.2  Design  Review  Committee 

It  is  the  responsibility  of  the  design  review  committee  to: 

a.  Plan  and  conduct  the  particular  design  review. 

b.  Advise  the  NOSC  program  manager  of  the  results  of  the  design  review. 

c.  Advise  the  Commander  and  technical  director  regarding  the  successful  attainment  of  a  program’s 
requirements  as  established  by  the  task  assignment  or  indicate  that  deviations  from  the  original 
requirements  are  known  and  acceptable. 

d.  Assist  the  Center  System  Safety  Office,  in  the  course  of  the  review,  by  identifying  the  critical 
system  safety  risks  and  appropriately  relating  these  to  the  design  requirements  and  the  safety 
measures  being  taken. 

e.  Verify  for  the  technical  director  that  adequate  producibility  and  design  evaluation  have  been 
attained  prior  to  release  to  production. 

f.  Submit  recommendations  to  the  technical  director  concerning  the  full  or  limited  production 
release  to  the  cognizant  project  agency  of  a  NOSC-developed  item,  subsystem,  or  system. 

g.  Review,  prior  to  release,  any  Center  letter  to  the  cognizant  project  agency  which  recommends 
release  to  production  and  states  any  limitations  or  compromises. 

17.5.3  Organization  of  the  Design  Review  Committee 

The  committee  membership  is  as  follows: 

Chairman *:  The  chairman  is  the  cognizant  department  head  of  the  subject  program. 


•For  design  reviews  for  minor  systems,  these  responsibilities  may  be  delegated  to  the  appropriate  division  heads  on 
a  case-by-case  basis  with  the  approval  by  the  Technical  Director,  Code  01. 


Assistant  to  the  Chairman :  A  person  selected  by  the  chairman  to  assist  the  chairman  and  project 
engineer  to  see  that  all  data  required  by  the  design  review  committee  is  acquired  prior  to  the  meeting, 
to  identify  action  items,  to  write  up  minutes  of  the  meeting,  and  to  follow  up  in  completing 
outstanding  issues  or  action  items. 

Permanent  Member *:  The  Head,  Engineering  and  Computer  Sciences  Department,  Code  90,  assures 
that  the  review  process  is  followed  and  that  recommendations  are  complete  and  rational. 

Permanent  Recorder :  Design  Review  Office,  Code  902,  defines  design  review  requirements,  assists 
in  selection  of  review  committee  members  with  the  assistant  to  the  chairman,  maintains  files  of 
Center  experts,  maintains  archives  of  program  reviews,  and  records  completion  of  outstanding 
action  items  as  forwarded  by  the  assistant  to  the  chairman. 

Review  Team  Members :  The  team  members  are  selected  from  across  the  Center  by  the  chairman 
and  Code  90,  and  invited  to  participate  by  the  assistant  to  the  chairman  via  channels.  These  members 
ought  to  include: 

Another  technical  department  head 

NOSC  technical  officer(s)  having  appropriate  technical  or  Fleet  backgrounds. 

Independent  technical  experts  from  the  scientific  and  engineering  departments,  individually  selected 
by  the  chairman  with  line  management  approval,  to  meet  the  particular  needs  of  each  design  review. 

Ex-Officio  Advisory  Members:  These  include  additional  technical  experts,  as  required. 


•For  design  reviews  for  minor  systems,  these  responsibilities  may  be  delegated  to  the  appropriate  division  heads  on  a  case-hv 
ease  basts  with  the  approval  by  the  Technical  Director.  Code  01. 


17-9 


CONTENTS 


Introduction  . 

References . . 

Outline . 

Summary . 

Technical  Manager  Development . 

Knowledge  and  Skills  Required  in  Project  Management  . 

Methods  to  Develop  Project  Management  Knowledge  and  Skills . 

On-the-Job  Training  . 

Short  Courses  Available . 

Long-Term  Project  Management  Training  Available . 

Planning  for  Program  Manager  Development . 

Advice  and  Counseling  Regarding  Development  of  Program  Managers 


SECTION  18 

FOLLOW-ON  TRAINING 
J.  Barhoum,  Code  142 

18.1  INTRODUCTION 

18.1.1  References 

None. 

18.1.2  Outline 

Introduction 

References 

Outline 

Summary 

Technical  Manager  Development 

Knowledge  and  Skills  Required  in  Project  Management 
Methods  to  Develop  Project  Management  Knowledge  and  Skills 
On-the-Job  Training 
Short  Courses  Available 

Long-Term  Project  Management  Training  Available 
Planning  for  Program  Manager  Development 

Advice  and  Counseling  Regarding  Development  of  Program  Managers 


18.1.3  Summary 

See  below. 


18.2  TECHNICAL  MANAGER  DEVELOPMENT 

This  Center  is  committed  to  the  development  of  successful  and  competent  program/project 
managers.  Previous  sections  of  this  handbook  detail  the  different  areas  in  which  a  technical  manager 
must  be  knowledgeable  and  highly  skilled.  In  this  section,  a  road  map  of  training  and  developmental 
information  is  presented  to  assist  you  in  acquiring  the  necessary  skills  and  abilities  to  become  a  competent 
technical  manager. 


18.3  KNOWLEDGE  AND  SKILLS  REQUIRED  IN  PROJECT  MANAGEMENT 


Potential  project  management  (PM)  knowledge  is  entensive,  and  skills  are  many  and  varied.  The 
technical  manager  must  know  the  science  and  engineering  used  in  the  development  task,  the  policies 
of  government  in  acquisition  management,  the  NOSC  method  of  doing  the  business  of  engineering 
management,  the  tools  to  ensure  that  what  was  envisioned  is  finally  produced  within  schedule  and  cost, 
and  personal  and  social  interactions  of  motivating  and  rewarding  task  team  members.  A  listing  of  these 
skills  and  knowledge  follows: 


PM  definition  and  responsibilities 

NOSC  policies  and  procedures 

DoD  and  Navy  acquisition  processes 

Project  formation 

Planning  and  control 

Budgeting  and  accounting 

Marketing 

Systems  engineering 

Safety 

Team  building 
Conflict  resolution 


Product  Assistance 
Contracting 
Design  review 
Test  and  evaluation 
Integrated  logistics  support 
Risk  assessment 
Warranty 
Documentation 
Negotiation  techniques 
Presentation  skills 
Writing  skills 


18.4  METHODS  TO  DEVELOP  PROJECT  MANAGEMENT  KNOWLEDGE  AND  SKILLS 

There  are  three  primary  methods  to  develop  project  management  skills:  on-the-job  training,  short 
courses,  and  long-term  academic  training.  Any  one  method,  by  itself,  is  not  the  best  way  to  achieve 
competence  as  a  project  manager.  A  mixture  of  the  three  methods  planned  over  a  period  of  several 
years  is  much  more  realistic.  An  individual  who  wishes  to  become  a  project  manager  needs  to  lay  out 
a  plan  that  uses  at  least  the  first  two  methods  to  acquire  the  knowledge  and  skills  that  are  not  yet 
possessed.  Let  us  examine  these  three  methods. 


18.4.1  On-The-Job  Training 

A  technical  staff  member  at  NOSC  usually  starts  as  a  member  of  a  small  project  team  and  deals 
with  a  more  senior  technical  member,  branch  head,  or  project  leader.  As  experience  grows,  larger  and 
more  complex  tasks  are  assigned  and  these  frequently  lead  to  becoming  a  key  member  of  a  greater 
and  more  complex  program  management  team.  The  individual  watches  the  technical  leader  and  sees 
how  this  person  goes  about  meeting  project  requirements. 

There  are  additional  opportunities  offered  to  technical  journeymen  to  take  extended  details  to 
Navy  Headquarters  Program  Management  Offices  to  support  the  project  work  that  is  being  performed 
by  NOSC  and/or  by  the  Navy  and  industry  development  organizations.  Occasionally,  details  to 
Washington  Headquarters  organizations  are  formally  announced  as  training  opportunities  under  the 
Navy  Scientist  Training  and  Exchange  Program  (NSTEP).  When  the  Center  receives  notification  of 
NSTEP  opportunities  these  are  distributed  to  the  technical  departments  for  interest  and  response. 
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18.4.2  Short  Courses  Available 


There  is  a  wide  assortment  of  short  courses  offered  on  the  knowledge  and  various  skills  required 
of  project  managers.  These  range  from  NOSC  presented  training,  Navy,  and  other  federally  sponsored 
courses  to  nongovernment  schools,  universities,  or  privately  owned  training  vendors. 

a.  NOSC  In-House  Courses 

Project  management  course 

Contracting  officer  technical  representative  (COTR)  course 

Presentations  and  briefings 

Technical  writing 

Financial  management 

Writing  statements  of  work 

Toastmasters 

b.  Government-Sponsored  Courses 

Navy 

Office  of  Personnel  Management  (OPM) 

Naval  Postgraduate  School 
Defense  Systems  Management  College 

c.  Nongovernment-Sponsored  Courses 

Private  vendors 

UCSD  Engineering  Management  Extension  Program 
UCLA  Engineering  Management 


18.4.3  Long-Term  Project  Management  Training  Available 

Although  these  types  of  opportunities  are  rarely  used  they  represent  an  important  resource  to  consider 
under  special  conditions.  If  a  technical  department  sees  a  need  for  trained  project  managers  within 
a  2-year  to  3-year  period  they  may  wish  to  accelerate  the  classroom  training  of  a  selected  staff  member 
by  encouraging  this  person’s  application  for  the  center’s  long-term  training  support  under  the  academic 
study  program.  Several  schools  have  excellent  programs.  The  following  programs  are  recommended 
in  order  of  relativity  to  NOSC  needs: 

Naval  Postgraduate  School 

Defense  Systems  Management  College 

USC  Systems  Management  Curricula 

UCLA  Masters  Program  in  Engineering  Management. 


18.5  PLANNING  FOR  PROGRAM  MANAGER  DEVELOPMENT 

Figure  18. 1  is  a  planning  guide  to  help  individuals  and  supervisors  be  assured  that  the  full  complement 
of  program  management  knowledge  and  skills  is  addressed.  It  is  recommended  that  this  guide  be  filled 
in  as  training  is  accomplished  and,  when  it  is  finally  completed,  that  the  technical  department  recognize 
the  achievement  of  the  person  in  some  formal  fashion,  i.e.  performance  award  recognition,  assignment 
to  project  leadership  role,  or  a  nonmonetary  form  of  recognition. 


18.6  ADVICE  AND  COUNSELING  REGARDING  DEVELOPMENT 
OF  PROGRAM  MANAGERS 

The  best  place  to  start  when  asking  questions  regarding  “how  do  I  become  a  project  manager?” 
is  with  your  line  supervisors  and  program  managers.  These  individuals  can  point  you  to  the  different 
methods  which  have  been  successfully  used  to  develop  their  experiences.  Many  will  comment  that  on- 
the-job  training  with  a  project  management  group  is  the  best  way;  they  will  also  recommend  course 
work  to  supplement  direct  experience. 

The  NOSC  Employee  Development  Office  advises  and  counsels  individuals  on  training  courses, 
academic  programs,  and  availability  of  NSTEP  assignments.  This  office  maintains  numerous  course 
descriptions  and  publicizes  in-house  courses  and  federal  government  training  opportunities;  the  staff 
will  strive  to  meet  identified  training  needs. 
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SECTION  19 
HUMAN  FACTORS 
C.  M.  Dean,  Code  441 


19.1  INTRODUCTION 

19.1.1  References 

MIL-STD-1472  C,  Human  Engineering  Design  Criteria  for  Military  Systems,  Equipment  and 
Facilities,  2  May  1981 

MIL-H-46855B,  Human  Engineering  Requirements  for  Military  Systems,  Equipment  and 
Facilities,  21  January  1979 

Human  Factors  Design  Handbook ,  Wesley  E.  \V  oodson,  McGraw-Hill  Book  Company,  1982 

Air  Force  Logistics  Command,  Human  Factor:  Engineering  Computer  Based  Instructional 
Course 

NOSC  TD  250,  Revision  A,  Suggestions  for  Designers  of  Navy  Electronic  Equipment,  May  1985 

19.1.2  Outline 

Introduction 

References 

Outline 

Summary- 

General 

Purpose 

Background 

The  Human  Being  as  System  Component 
Optimum  System  Performance 
Benefits  of  Human  Factors  Engineering 
HFE  Methodological  Tools  and  Design  Aids 
Allocation  of  Function  or  Who  Does  What? 

Conclusion 

Problems  with  Users 
Problems  with  Engineers 
Problems  with  Behavioral  Scientists 
Still  More  Problems 
The  Final  Word 

Appendix  19A  Auxiliary  Human  Factors  Information 

19.1.3  Summary 

See  below.  Also  consider  the  words  of  Albert  Einstein,  “Concern  for  man  himself  and  Ins  late 

must  alwa\s  form  the  chief  interest  of  all  technological  endeavors.” 
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19.2  GENERAL 


Human  Factors,  Human  Engineering,  Human  Factors  Engineering,  Man-Machine  Interface, 
Human-Computer  Interface,  User-Computer  Interface,  Man-Machine  Technology,  Ergonomics.  These 
are  all  terms  which  are  used,  sometimes  interchangeably,  to  indicate  those  investigators  who  treat  the 
human  user  as  part  of  the  system  design  process.  Human  factors  engineering,  or  HFE,  is  the  practice 
of  designing  products  so  that  a  human  being  can  use  these  products  for  their  designed  purposes,  operate 
them  easily,  service  them,  and  support  them  in  situ.  All  of  this  should  be  accomplished  with  a  minimum 
of  stress  and  a  maximum  of  efficiency.  In  simple  terms,  human  factors  are  characteristics  of  people 
—  characteristics  such  as  size,  shape,  ability  to  see  and  hear,  strength,  intelligence,  temperament, 
forgetfulness,  and  weakness.  Such  characteristics  —  the  human  factors  —  must  be  taken  into  account 
so  that  human  beings  and  things  made  for  their  use  will  go  well  together. 

19.2.1  Purpose 

The  purpose  of  this  section  is  to  provide  NOSC  program  managers  with  an  understanding  of  the 
need  for  and  benefits  of  including  human  factors  expertise  as  part  of  the  program  design  team  and 
to  provide  some  useful  references  to  consult  for  answers  to  human  factors  questions. 


19.2.2  Background 


A  report  by  the  Navy  Research  Advisory  Committee  on  Man-Machine  Technology  in  the  Navy, 
December  1980,  stated: 

The  human  element  has  become  the  most  critical,  most  problematic  and  most  costly  component 
of  the  Navy.  Meanwhile,  increasingly  complex  hardware  systems  are  being  developed  and  procured 
for  fleet  use. 

Given  present  trends,  the  Navy  will  find  itself  unable  to  operate  and  maintain  its  systems,  in  either 
the  short  or  long  term,  with  the  numbers  of  skilled  personnel  necessary  for  effective  mission 
accomplishment. 

A  Report  to  the  Congress  of  the  United  States  by  the  Comptroller  General  in  January,  1981, 
“Effectiveness  of  U.S.  Forces  Can  Be  Increased  Through  Improved  Weapon  System  Design”  concluded: 

There  are  indications  that  human  ineptitude  or  poor  human  reliability  may  cause  over  50  percent 
of  all  weapon  system  failures.  The  increasingly  complicated  nature  of  modern  military  systems 
together  with  internal  military  personnel  problems  suggests  that  human-induced  errors  both  in 
operations  and  maintenance  could  also  increase  unless  more  attention  is  paid  to  this  problem  during 
design  and  development.  Weapon  systems  designs  have  been  dictating  manpower  requirements. 
What  is  needed  is  a  continuing  interface  between  the  system  designers  and  the  manpower  planners 
with  manpower  requirements  influencing  system  design  and  vice  versa. 

If  the  design  of  systems  is  to  adequately  consider  all  the  human  limitations  (including  skill  levels, 
proficiency,  availability,  environmental  stress,  and  fatigue),  military  specifications,  standards,  and 
Handbooks  must  address  these  factors.  Existing  documents  do  not.  Also,  common  methodologies 
and  sources  of  data  are  needed  to  forecast  skill  levels  of  potential  military  personnel  5  to  10  years 
in  the  future.  This  information,  which  would  be  extremely  valuable  to  system  designers  and  testers, 
is  currently  not  available. 
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Finally,  there  does  not  appear  to  be  sufficient  emphasis  on  testing  systems  from  a  human  reliability 
standpoint  particularly  in  the  developmental  stages  of  the  acquisition  process.  This  could  result 
in  design  errors  requiring  expensive  modifications  after  the  system  is  deployed. 

An  Army  technical  memorandum  indicated  the  importance  of  the  operator’s  and  maintainer’s 
relationship  to  an  item  of  hardware. 

“People  are  the  only  responsible  agents  in  the  system.  No  matter  how  small  the  roles  assigned 
to  people,  they  are  responsible  roles.  People  determine  whether  the  system  is  ready  to  operate, 
what  it  is  to  do,  how  and  when  it  is  to  do  it,  when  and  what  variations  in  performance  are  to 
occur,  and  what  constitutes  adequate  or  complete  performance.  People  decide,  control,  guide  change, 
and  evaluate.  They  are  expected  to  anticipate,  detect,  compensate  for,  and  explain  any  undesirable 
variations  in  performance.  And  their  errors  assume  a  significance  commensurate  with  their 
responsibilities.” 

These  relatively  recent  reports  point  out  the  importance  of  the  human  being  as  a  system  component 
and  reiterate  problems  which  have  existed  as  long  as  human  beings  have  been  required  to  use  tools 
to  perform  work.  The  need  for  human  factors  as  a  discipline  escalated  during  World  War  II  when 
the  armed  forces  recruited  thousands  of  men  from  all  walks  of  life  and  faced  the  problems  of  training 
them  to  perform  unfamiliar  tasks  quickly  with  military  hardware. 

While  human  beings  and  machines  have  been  around  for  a  long  time,  the  profession  of  human 
factors  engineer  is  relatively  young,  and  it  is  generally  made  up  of  people  from  varying  disciplines  such 
as  cognitive  and  industrial/organizational  psychology,  physiology,  engineering,  (all  types),  industrial 
design,  computer  science,  anthropology,  sociology,  systems  analysis,  biomechanics,  operations  research, 
and  business  and  management  science.  Human  factors  is  truly  an  interdisciplinary  profession  with  the 
common  purpose  of  enhancing  the  performance  of  the  entire  system  while  including  the  human  user 
as  an  integral  part  of  that  system  design. 

Human  factors  engineering  is  a  discipline  born  of  crisis.  After  every  system  failure  resulting  in 
catastrophic  loss,  from  World  War  II  pilots  mistaking  landing  gear  levers  for  throttles  to  the  Three 
Mile  Island  and  Chernobyl  nuclear  disasters  and  to  the  space  shuttle  explosion,  accident  investigators 
have  tried  to  pinpoint  the  causes  of  failure  and  in  many  instances  have  concluded  that  the  system  failed 
because  of  human  error.  Whenever  human  beings  are  part  of  the  system  there  will  be  human  errors. 
The  task  of  designers  is  to  predict  what  those  are  likely  to  be  and  take  appropriate  measures  to  preclude 
their  occurrence.  System  designers  are  most  successful  in  this  endeavor  when  they  adopt  a  human- 
centered  design  philosophy  and  enlist  the  aid  of  human  factors  experts.  The  tasks  of  the  human  factors 
engineer  are  then  to  1)  evaluate  the  human  being  as  a  system  component  and  his  or  her  contribution 
to  the  total  system;  and  2),  to  influence  the  selections  among  design  alternatives  as  they  relate  to  people. 

19.3  THE  HUMAN  BEING  AS  SYSTEM  COMPONENT 

In  evaluating  the  human  user  as  a  primary  system  component,  the  human  factors  engineer  will 
take  into  account  such  human  capabilities  as  the  following: 

a.  Information  sensors 

b.  Information  processors 

c.  Response  mechanisms 

d.  Their  unique  properties  as  human  beings. 
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Human  beings  as  system  components  exhibit  variability  in: 

a.  Dimensions 

b.  Perceptions 

c.  Reactions 

d.  Tolerances. 

All  of  these  human  variables  are  well  documented  in  the  anthropometric  literature.  The  Human  Factors 
Design  Handbook  by  Woodson  has  several  chapters  devoted  to  the  subject.  However,  it  is  well  to  note 
that  even  among  “homogeneous”  populations,  for  example  all  Navy  males  between  17  and  24,  the 
range  of  dimensions  can  be  considerable.  Variability  is  usually  expressed  in  terms  of  percentiles.  Most 
designs  should  accommodate  a  wide  range,  generally  the  2nd  to  98th  or  5th  to  95th  percentile  ranges. 
A  piece  of  equipment  which  is  designed  for  the  “average”  reach,  the  50th  percentile,  will  be  out  of 
reach  for  50  percent  of  the  intended  users.  “Average”  is  a  statistical  term  which  applies  only  to  groups. 
Most  people  will  fall  into  the  “average”  range  on  only  a  few  variables. 

Variability  of  human  dimensions  can  best  be  handled  for  a  wide  range  of  people  by  employing 
adjustability.  There  are  certain  applications,  space  suits  for  example,  where  it  will  be  necessary  to 
customize  the  design  for  each  segment  of  the  population.  Designing  for  the  mythical  average  user  is 
sometimes  acceptable,  but  usually  is  preceded  by  extensive  research  and  iterative  design.  An  example 
of  a  design  for  an  average  user  might  be  a  telephone  or  computer  keyboard. 

When  designing  prototype  tests,  the  following  key  dimensions  in  human  performance  should  be 
considered: 

a.  Speed  versus  accuracy 

b.  Skilled  versus  unskilled 

c.  Performance  standards  and  criteria 

d.  Hawthorne  effect  and  related  “artifacts” 

e.  Individual  differences 

f.  Training. 

Performance  standards  and  criteria  should  be  specified  before  the  test  is  run.  The  use  of  control  and 
experimental  groups  of  subjects  should  be  the  rule,  otherwise  the  test  results  will  be  contaminated  by 
the  subjects  having  been  given  previous  training.  Training  effects  can  cause  confounding  of  test  results 
due  to  either  positive  or  negative  transfer  of  training.  An  example  of  positive  transfer  of  training  would 
be  the  elevated  score  on  a  word  processing  test  that  a  subject  who  had  previous  experience  on  a  typewriter 
might  obtain.  Negative  transfer  of  training  would  occur  if  a  person  had  been  trained  to  operate  a  shift 
lever  with  his  left  hand  and  the  new  design  required  right  hand  operation. 

The  Hawthorne  effect  is  named  after  some  studies  which  were  done  in  an  industrial  plant  in 
Hawthorne,  Pennsylvania  some  years  ago.  Researchers  told  factory  workers  that  they  were  studying 
productions  rates  and  that  certain  workers  would  be  selected  to  participate.  They  observed  that  production 
rates  went  up  even  without  making  any  changes  to  the  workers  environment.  In  fact,  when  lighting 
was  increased,  production  went  up,  and  when  lighting  was  decreased  production  also  went  up.  The 
moral  of  this  story  is  that  certain  behaviors  may  occur  merely  because  people  are  aware  that  they  are 
being  watched.  Human  factors  engineers  will  be  able  to  help  design  tests  to  guard  against  such  artifacts 
and  avoid  wasted  time  and  money. 


19.4  OPTIMUM  SYSTEM  PERFORMANCE 


System  performance  is  defined  as  the  interaction  between  human  behavior  and  the  performance 
of  nonhuman  system  elements  such  as  equipment,  procedures,  and  environmental  support  facilities. 
Human  behavior  in  the  system  includes  not  only  the  human  operator  or  user,  but  the  behavior  of  those 
others  who  have  to  maintain  the  system  in  operable  condition  and  still  others  who  are  involved  in 
developing  the  various  supporting  documents  and  equipment  and  the  training  courses  and  materials. 
It  is  well  to  note  that  improvements  in  human  performance  may  or  MAY  NOT  affect  system  performance. 
Therefore,  systems  design  is  creating  optimum  system  performance  by  matching  and  integrating  people, 
processes,  and  materials  . . .  not  necessarily  optimizing  all  three  factors. 


19.5  BENEFITS  OF  HUMAN  FACTORS  ENGINEERING 


When  human  factors  concerns  are  identified  early  in  the  design  cycle  and  appropriate  alternatives 
are  selected  with  the  user  in  mind  as  an  integral  system  component,  the  resultant  benefits  include  the 
following: 

a.  Increased  productivity  and  performance 

b.  Minimized  operator  stress  and  error 

c.  Reduced  skill  level  requirements 

d.  Improved  system  reliability 

e.  Improved  marketability  and  longevity. 

The  consequences  of  not  considering  human  factors  issues  during  system  design  can  be  disastrous  because 
human  errors  are  inevitable  as  long  as  humans  are  operators,  maintainers,  and  trainers.  Human 
capabilities  do  not  equal  human  characteristics.  The  fact  that  humans  are  capable  of  discerning  a  warning 
light  or  alarm  does  not  guarantee  that  they  will  take  any  action  if  false  alarms  are  frequent.  Human 
adaptation  is  never  free;  it  always  comes  at  some  cost  which  may  not  be  the  concern  of  the  program 
manager,  but  ultimately  will  contribute  to  the  system  total  life-cycle  costs  by  requiring  expensive  retrofits, 
special  support  equipment,  additional  training,  or  production  of  additional  spare  parts.  While  it  is 
true  that  people  are  the  most  flexible  element  in  system  design,  they  are  the  most  difficult  to  redesign. 
The  costs  of  poor  design  are  tremendous  in  terms  of  personnel  selection,  training,  and  adaptability. 


19.6  HFE  METHODOLOGICAL  TOOLS  AND  DESIGN  AIDS 


In  addition  to  the  human  perception,  reaction,  tolerance,  and  anthropometric  data  bases  which 
human  factors  experts  have  compiled  over  the  years,  there  are  other  specific  methodological  tools  which 
are  employed  to  identify  human  factors  issues  in  system  design. 

Task  analysis  is  a  means  of  breaking  down  operations  into  smaller  units  in  order  to  determine 
whether  those  functions  should  be  correctly  allocated  to  the  human  user  or  should  be  automated. 

Job  analysis  is  a  means  of  determining  how  many  tasks  are  incorporated  into  each  job  unit  and 
then  taking  a  critical  look  at  the  tasks  within  each  job  to  determine  whether  or  not  it  is  beneficial  to 
both  human  beings  and  the  system  to  group  tasks  that  way. 

Systems  analysis  is  a  means  of  looking  at  the  entire  system,  including  the  human  user’s  contribution, 
to  determine  how  all  parts  of  the  system  work  together,  whether  any  part  is  overloaded  or  underused, 
and  especially  to  consider  the  affects  of  those  system  components  which  have  been  heretofore  neglected. 
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Human  performance  research  is  being  carried  out  at  many  government  and  private  laboratories 
throughout  the  country  and  the  world.  This  research  unfortunately  lags  behind  recent  technological 
leaps,  especially  in  the  user-computer  interface  realm.  Researchers  may  be  involved  in  testing  prospective 
users  on  particular  aspects  of  user-interface  design  for  something  like  a  new  radar  operator’s  work 
station  or  they  may  be  doing  more  basic  research  into  generic  questions;  for  example,  in  determining 
whether  a  trackball  or  a  mouse  provides  the  user  with  more  positive  feedback  and  control.  With  the 
vast  diversity  in  equipment  and  systems  being  designed,  just  in  the  Navy,  it  is  unlikely  that  any  design 
will  be  able  to  get  by  without  having  some  human  performance  research  done.  On  the  other  hand, 
there  is  a  large  body  of  published  literature  on  experiments  which  have  already  been  performed,  so 
it  will  not  be  necessary  in  most  cases  to  do  basic  research,  but  will  be  possible  to  build  on  the  results 
already  achieved  by  others  to  shorten  the  amount  of  necessary  human  performance  testing. 

Standards,  criteria,  and  checklists  are  often  most  helpful.  A  list  of  the  more  pertinent  of  these 
to  Navy  systems  is  included  under  the  reference  section.  A  few  tables  and  a  questionnaire  are  included 
at  the  end  of  this  article  to  aid  designers  in  allocation  of  function  and  ensure  they  have  addressed  all 
of  the  major  human  factors  issues. 

The  last  of  the  tools  and  design  aids  is  anecdotal  wisdom.  This  category  takes  in  all  the  knowledge 
that  has  been  gathered  too  recently  to  appear  in  publication,  maybe  as  recently  as  the  last  project  the 
human  factors  engineer  worked  on.  HFE  people  generally  keep  informed  about  the  latest  developments 
by  attending  conferences,  reading  periodicals,  talking  to  colleagues,  and  doing  their  own  research.  This 
category  has  taken  on  added  significance  with  the  explosion  in  new  features  being  invented  almost 
daily  for  computer  users. 


19.7  ALLOCATION  OF  FUNCTION  OR  WHO  DOES  WHAT? 

After  performing  a  task  analysis  it  will  be  necessary  to  make  some  decisions  about  allocating 
functions  either  to  the  human  user  or  to  some  other  system  component.  In  making  these  decisions, 
designers  must  consider  what  people  WILL  do  (characteristic  performance),  not  what  they  can  do 
(capability).  Keep  in  mind  the  example  of  false  alarms  discussed  above.  Most  contemporary  decisions 
involve  the  LEVEL  and  NATURE  of  semiautomation  that  is  appropriate  as  opposed  to  traditional 
manned  versus  unmanned  systems.  The  space  shuttle  was  uniquely  qualified  to  put  people  in  orbit, 
sustain  life  while  certain  tasks  could  be  carried  out,  and  safely  return  to  the  earth.  It  was  an  almost 
ideal  testbed  for  zero-G  experimentation.  The  fact  that  it  could  also  launch  satellites  contributed  to 
the  decision  to  develop  an  ambitious  launch  schedule  to  help  it  “pay  its  way”.  A  much  better  decision 
in  the  case  of  satellite  launch,  as  it  now  appears,  would  have  been  to  allocate  that  function  to  an  unmanned 
spacecraft.  Likewise,  for  many  other  less  sophisticated  systems,  the  fact  that  it  is  possible  for  either 
a  machine  or  a  human  being  to  perform  a  function  does  not  necessarily  require  it.  The  tables  included 
at  the  end  of  this  article  compare  and  contrast  human  and  machine  capabilities  and  limitations  and 
should  be  used  to  help  decide  who  ultimately  does  what. 


19.8  CONCLUSION 

Human  factors  engineering,  encompassing  a  body  of  knowledge  about  human  behavior  in  systems, 
is  a  multidisciplinary  field  that  advocates  enhancing  system  performance,  while  including  the  human 
user  as  a  primary  system  component,  and  provides  tools  and  design  aids  to  assist  system  designers 
in  identifying  and  overcoming  human  factors  problems. 
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All  this  effort  has  still  not  succeeded  in  solving  all  the  problems.  System  designers  should  be  aware 
of  some  of  the  lingering  and,  perhaps,  eternal  problems  listed  below  . 

19.8.1  Problems  with  Users 

a.  Don’t  care  about  the  elegance,  sophistication,  or  complexity  of  the  design  —  only  what  it 
can  (or  can’t)  do  for  them 

b.  Will  do  the  unexpected 

c.  Will  base  their  conception  of  the  system  on  inadequate  knowledge 

d.  Won’t  ask  for  help  when  they  need  it 

e.  Will  fail  to  observe  the  prominently  displayed  instructions 

f.  Quickly  develop  habits  which  are  hard  to  change 

g.  Can  be  “suspicious”  of  the  system 

h.  Sometimes  fear  that  they  will  break  the  system 

19.8.2  Problems  with  Engineers 

a.  They  assume  too  much  about  users. 

b.  They  tend  to  focus  on  hardware-oriented  system  criteria  (e.g.  processing  speed) 

c.  They  “get  attached”  to  hardware  (also  software) 

d.  Many  will  never  use  the  systems  that  they  design 

e.  They  are  suspicious  of  “soft”  sciences  like  psychology  (this  isn’t  always  inappropriate!). 

19.8.3  Problems  with  Behavioral  Scientists 

a.  Research  reports  often  lack  design-relevant  interpretations  and  recommendations. 

b.  Laboratory  studies  may  not  generalize  to  an  operational  context. 

c.  Lack  of  familiarity  with  hardware  systems  can  lead  to  the  study  of  “unreal”  variables. 

d.  Obsession  with  advanced  statistical  methods  can  lead  to  intense  scrutiny  of  trivial  factors. 

19.8.4  Still  More  Problems 

a.  Standards  are  adopted  for  political  reasons. 

b.  Economic  incentives  often  promote  the  status  quo. 

c.  There  is  a  tendency  to  translate  old  concepts  and  models  to  new  technology  (this  can  be  limiting). 

d.  Creative  problem-solving  requires  diverse  backgrounds,  including  people  who  know  L1TT1  T 
OR  NOTHING  ABOUT  THE  TECHNICAL  LIMITATIONS. 

e.  “Implicit  assumptions”  can  plague  otherwise  excellent  designs. 
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19.8.5  The  Final  Word 

Successful  system  design  places  early  focus  on  users  and  tasks,  employs  empirical  measurements 
of  human  performance,  and  iterates  the  design  in  order  to  achieve  optimum  system  performance.  Human 
factors  expertise  should  be  applied  in  system  planning,  system  design  and  implementation,  system 
evaluation,  and  system  modification.  Unfortunately,  human  factors  are  often  applied  only  where 
government  requires  it,  or  too  early  in  system  planning,  or  too  late  in  system  design,  or,  the  ultimate, 
when  everything  else  has  been  tried.  There  are  enough  stories  circulating  about  equipment  poorly 
designed,  inoperable,  unsupportable,  lacking  standardization,  with  poor  accessibility,  and  poor  man- 
machine  interface  to  keep  human  factors  experts  going  for  years  trying  to  clear  up  the  problems.  The 
alternative  is  for  system  designers  and  program  managers  to  avail  themselves  of  the  human  factors 
expertise  resident  at  NOSC  to  ensure  that  they  will  become  one  of  the  success  stories.  Or,  in  the  words 
of  that  preeminent  scientist,  Albert  Einstein, 


“Concern  for  man  himself  and  his  fate  must  always  form  the  chief  interest  of  all  technological 
endeavors.” 
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Auxiliary  Human  Factors  Information 


The  following  items  are  included  in  the  appendix: 

Human  Factors  Engineering  (HFE)  Computer-Based  Instructual  (CB1)  Course 
Log-On  Procedures  for  Introduction  to  Human  Factors  Engineering 

Human  Factors  in  Engineering  and  Design,  Ernest  J.  McCormick,  McGraw-Hill,  Inc.,  1976 
(an  excerpt) 

Human  Limitations  and  Machine  Alternatives 
Allocation  of  Function/Allocation  of  Function  Procedures 
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HUMAN  FACTORS  ENGINEERING  (HFE) 

COMPUTER-BASED  INSTRUCTIONAL  (CBI)  COURSE 

The  HFE  CBI  course  is  an  adaptation  of  the  ARMY/NAVY  self-paced  HFE  text  developed  by 
Brogan,  et.  ai.  of  the  ARMY  HEL  in  1981. 

The  course  objectives  are: 

1.  An  understanding  of  common  HFE  terms 

2.  An  awareness  of  sources  of  HFE  information 

3.  An  ability  to  integrate  HFE  into  a  development  or  modification  program  with  minimum 
guidance  and  direction  from  experienced  HFE  personnel 

4.  An  ability  to  determine  HFE  requirements 

5.  An  understanding  of  the  kinds  of  factors  and  forces  which  affect  human  performance 

6.  An  awareness  of  HFE  test  methods 

7.  An  awareness  of  human  performance  reliability  factors 

8.  An  awareness  of  time  and  error  performance  measures 

9.  An  understanding  of  the  major  HFE  techniques 

10.  A  familiarity  with  task  analyses 

11.  An  awareness  of  the  relationship  between  HFE  and  reliability 

12.  An  ability  to  apply  HFE  standards,  specifications,  and  references. 

NONPROPRIETARY  SOFTWARE 

The  applications  software  is  wholly  developed  and  owned  by  the  Air  Force  Logistics  Command. 
There  is  nothing  proprietary  in  the  software.  Therefore,  it  can  be  provided,  free,  to  any  DoD  agency 
that  finds  a  need  to  implement  CBI  courseware.  Given  the  currently  high  licensing  fees  for  commercially 
available  CBI  software,  our  CBI  software  may  be  able  to  save  one  of  your  development  programs 
a  considerable  amount  of  money. 

COURSE  WORK 

The  course  can  be  accessed  by  almost  any  computer  terminal  that  has  a  300  or  1200  BAUD  telephone 
modem.  As  long  as  our  computer  resources  are  not  overlooked  by  too  many  students,  then  we'll  provide 
an  accessible  HFE  CBI  course  for  all  DoD  personnel  and  contractors  with  active  DoD  contracts.  The 
phone  company,  however,  will  charge  you  for  a  commercial  long  distance  phone  call  if  you  use  a 
commercial  long  distance  line  to  hook-up  your  modem. 
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LOG-ON  PROCEDURES  FOR 
INTRODUCTION  TO  HUMAN  FACTORS  ENGINEERING 


These  instructions  explain  procedures  to  access  the  HFE  self-paced  course  on  the  AFLC  computer. 
Almost  any  computer  terminal  can  be  used  as  a  dumb  terminal  to  access  the  course.  Most  terminals 
have  switches  on  them  that  can  be  set  to  various  configurations.  Your  terminal  must  be  configured  with: 


BAUD  RATE 

DUPLEX 

PARITY 

CARRIAGE  RETURN 
CHARACTER  TYPE 


300  OR  1200  (ASYNCHRONOUS  LINE) 

HALF  (OR  FULL  IF  SELF  ECHO  IS  ON) 

EVEN 

WITHOUT  LINE  FEED 

7  BIT  ASCII  PLUS  1  BIT  EVEN  PARITY,  1  START  BIT, 
1  STOP  BIT 


Phone  the  AFLC  computer  at: 


1200  BAUD  RATE  LINE  -  AV  787-8243  or  COMMERCIAL  (513)  257-8243 

300  BAUD  RATE  LINE  —  AV  787-8247/53/57/65  or  COMMERCIAL  (513)  257-8247/53/57 

After  the  carrier  tone  is  present  on  the  phone  line  and  the  telephone  receiver  is  connected  to  your 
computer  modem,  then  you  must  enter  each  of  the  following  commands  with  a  carriage  return  after 
each  one: 


WHEN  THE  COMPUTER  DISPLAYS,  THEN  THE  STUDENT  ENTERS: 

?  ZW..TSS  (or  use  VD„TSS) 

USER  ID—  HUMANSFACTORS 

PROBLEM  NUMBER  WP1906 

*  FRN  HFE/TUTOR.E 

It  is  important  that  the  above  commands  are  entered  exactly  as  shown  above,  i.e.  do  not  insert 
spaces  where  none  are  shown.  Use  the  @  symbol  to  internally  delete  characters  that  you  have  misspelled. 

After  entering  the  last  command  above,  the  computer  will  require  about  20  to  40  seconds  to  load 
and  compile  the  program.  Then,  the  computer  will  lead  you,  step  by  step,  through  the  self-paced  course 
by  listing  information  to  you  and  asking  questions. 

If  you  use  an  autovon  line  to  link  your  terminal  to  the  AFLC  computer,  it  is  always  subject  to 
interruption.  If  this  occurs,  the  computer  will  remember  where  you  were  in  the  lesson  and  will  restart 
you  at  the  point  that  you  were  interrupted.  Have  patience  and  start  the  above  log-on  procedure  again. 
Use  a  commercial  phone  line  if  problems  with  autovon  persist. 

Your  organization  will  not  be  charged  for  the  AFLC  computer  time.  The  course  is  t  ree  to  all  DoD 
employees  and  contractor  personnel  with  active  DoD  contracts.  The  phone  company,  however,  will 
charge  you  for  a  commercial  line,  if  used.  A  training  certificate  will  be  sent  to  you  when  you  finish 
all  the  lessons. 


To  exit  the  program: 

For  teletype  terminals,  depress  the  interrupt  button  and  follow  the  instructions  that  are  listed  to  you. 

For  CRT  terminals,  enter  the  letter  B  whenever  a  question  is  asked  of  you.  Then  follow  the 
instructions  that  are  listed  to  you. 

IF  YOU  HAVE  PROBLEMS,  CONTACT  NAT  DAVIS  —  AV  787-5571  OR  (513)  275-5571 
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Human  Factors  in  Engineering  and  Design, 

Ernest  J.  McCormick,  McGraw-Hill  Inc.,  1976  (an  excerpt) 

The  questions  given  below  should  be  viewed  as  they  would  be  relevant  to  the  ultimate  user 
population.  The  fulfillment  of  one  objective  may  of  necessity  be  at  the  cost  of  another. 

In  a  general  sense  these  questions  may  serve  as  “reminders”  of  some  of  the  human  factors 
considerations  that  should  be  taken  into  account  in  the  design  process. 

1.  What  are  the  functions  that  need  to  be  carried  out  to  fulfill  the  system  objective? 

2.  If  there  are  any  reasonable  options  available,  which  of  these  should  be  performed  by  human 
beings? 

3.  For  a  given  function,  what  information  external  to  the  individual  is  required?  Of  such 
information,  what  information  can  be  adequately  received  directly  from  the  environment, 
and  what  information  should  be  presented  through  the  use  of  displays? 

4.  For  information  to  be  presented  by  displays,  what  sensory  modality  should  be  used? 
Consideration  should  be  given  to  the  relative  advantages  and  disadvantages  of  the  various 
sensory  modalites  for  receiving  the  type  of  information  in  question. 

5.  For  any  given  type  of  information,  what  type  of  display  should  be  used?  The  display  generally 
should  provide  the  information  when  and  where  it  is  needed.  These  considerations  may  take 
into  account  the  general  pe  of  display,  the  stimulus  dimension  and  codes  to  be  used,  and 
the  specific  features  of  the  display.  The  display  should  provide  for  adequate  sensory 
discrimination  of  the  minimum  differences  that  are  required. 

6.  Are  the  various  visual  displays  arranged  for  optimum  use? 

7.  Are  the  information  inputs  collectively  within  the  reasonable  bounds  of  human  information¬ 
receiving  capacities? 

8.  Do  the  various  information  sources  avoid  excessive  time-sharing? 

9.  Are  the  decisionmaking  and  adaptive  abilities  of  human  beings  appropriately  utilized? 

10.  Are  the  decisions  to  be  made  at  any  given  time  within  the  reasonable  capability  limits  of 
human  beings? 

11.  Granting  that  aspects  of  some  systems  will  be  automated,  is  the  basic  control  of  the  system 
that  of  the  individual? 

12.  When  physical  control  is  to  be  exercised  by  an  individual,  what  type  of  control  device  should 
be  used? 

13.  Is  each  control  device  easily  identifiable? 

14.  Is  the  operation  of  each  control  device  compatible  with  any  corresponding  display,  and  with 
common  human  response  tendencies? 

15.  Are  the  operational  requirements  of  any  given  control  (as  well  as  of  the  controls  generally) 
within  reasonable  bounds?  The  requirements  for  force,  speed,  precision,  etc.,  should  be  within 
limits  of  virtually  all  persons  who  are  to  use  the  system.  The  man-machine  dynamics  should 
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>o  capitalize  on  human  abilities  that,  in  operation,  the  devices  meet  the  specified  system 
requirements. 

16.  Are  the  control  devices  arranged  conveniently  and  for  reasonably  optimum  use? 

17.  If  there  is  a  communication  network,  will  the  communication  How  avoid  overburdening  the 
individuals  involved? 

18.  Are  the  various  tasks  to  be  done  grouped  appropriately  into  jobs? 

19.  Do  the  tasks  which  require  time-sharing  avoid  overburdening  any  individual  or  the  system? 
Particular  attention  needs  to  be  given  to  the  possibility  of  overburdening  in  emergencies. 

20.  Is  there  provision  for  adequate  redundancy  in  the  system,  especially  of  critical  functions? 
Redundancy  can  be  provided  in  the  form  of  backup  or  parallel  components  (either  men  or 
machines). 

21 .  Are  the  jobs  of  such  a  nature  that  the  personnel  to  perform  them  can  be  trained  to  do  them? 

22.  If  so,  is  the  training  period  expected  to  be  within  reasonable  time  limits? 

23.  Do  the  work  aids  and  training  complement  each  other? 

24.  If  training  simulators  are  used,  do  they  achieve  a  reasonable  balance  between  transfer  of 
training  and  costs? 

25.  Is  the  work  space  suitable  for  use  by  the  range  of  individuals  who  will  use  the  facility? 

26.  Are  the  environmental  conditions  (temperature,  illumination,  noise,  etc.)  such  that  they  permit 
satisfactory  levels  of  human  performance  and  provide  for  the  physical  well-being  of 
individuals? 

2'.  In  any  evaluation  or  test  of  the  system  (or  components)  does  the  system  performance  meet 
the  desired  performance  requirements? 

28.  Does  the  system  in  its  entirety  provide  reasonable  opportunity  for  the  individual(s)  involved 
to  experience  some  form  and  degree  of  self-fulfillment  and  to  fulfill  some  of  the  human  values 
that  we  should  all  like  to  have  the  opportunity  to  fulfill  in  our  daily  lives? 

29.  Does  the  system  in  its  entirety  contribute  generally  to  the  fulfillment  of  reasonable  human 
values?  In  the  case  of  systems  with  identifiable  outputs  of  goods  and  services,  this  consideration 
would  apply  to  those  goods  and  services.  In  the  case  of  systems  that  relate  to  our  life  space 
and  everyday  Living,  this  consideration  would  apply  to  the  potential  fulfillment  of  those  human 
values  that  are  within  the  reasonable  bounds  of  our  civilization. 

In  the  resolution  of  these  and  other  kinds  of  human  factors  considerations,  one  should  draw  upon 
whatever  relevant  information  is  available.  This  information  can  be  of  different  types,  including  principles 
that  have  been  developed  through  experience  or  research,  sets  of  normative  data  (such  as  frequency 
distributions  of,  say,  body  size),  sets  of  factual  data  of  a  probability  nature  (such  as  percentage  of 
signals  that  are  detected  under  specified  conditions),  mathematical  formulas,  tentative  theories  of 
behavior,  hypotheses  that  have  been  suggested  by  research  investigations,  and  even  the  general  knowledge 
acquired  through  everyday  experience. 
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HUMAN  LIMITATIONS  AND  MACHINE  ALTERNATIVES 


HUMAN 

Is  a  poor  monitor  of  infrequent  events  or  of  events 
which  occur  over  a  long  period  of  time 

Has  a  limited  channel  capacity 

Is  subject  to  coriolis  effects,  motion  sickness, 
disorientation,  etc. 

Has  extremely  limited  short  term  memory  for  fac¬ 
tual  material 

Is  not  well  suited  to  data  coding,  amplification, 
or  transformation  tasks 

Performance  is  degraded  by  fatigue  and  boredom 

Performance  is  degraded  by  long  duty  periods, 
repetitive  tasks,  and  cramped  or  unchanged 
positions 

Becomes  saturated  quickly  in  terms  of  the  number 
of  things  that  can  be  done  and  the  duration  of 

'.ne  effort 

May  introduce  errors  by  misidentification, 
reintegration,  or  closure 

Expectation  or  cognitive  set  may  lead  an  operator 
to  see  what  he  or  she  expects  or  wants  to  see 

Much  human  mobility  is  predicated  and  based  on 
gravity  relationships 

Is  adversely  affected  by  g  forces 

Can  generate  only  relatively  small  forces  and  can¬ 
not  exert  large  forces  for  very  long  or  very 
smoothly 

Generally  requires  a  review  or  rehearsal  period 
before  making  decisions  based  on  items  in 
memory 

When  performing  a  tracking  task,  requires  fre¬ 
quent  reprogramming;  does  best  when  changes  are 
under  3  rad  s 

Has  a  build-in  response  latency  of  about  200ms 
in  a  go-no-go  situation 


MACHINE 

Can  be  constructed  to  detect  infrequent  events  or 
events  which  occur  frequently  over  a  long  period 
of  time  reliably 

May  have  as  much  channel  capacity  as  can  be  af¬ 
forded 

Is  not  subject  to  these  effects 

May  have  as  much  short  term  (buffer)  memory 
as  can  be  afforded 

Is  well  suited  to  these  tasks 

Performance  is  degraded  only  by  wearing  out  or 
by  lacking  of  calibration 

Is  less  affected  by  long  duty  periods  and  performs 
repetitive  tasks  well;  some  may  be  restricted  by 
position 

Can  do  one  thing  at  time  so  fast  that  it  seems  to 
do  many  things  at  once,  for  a  long  period  of  time 

Uses  these  processes 

Does  not  exercise  these  processes 

May  be  built  to  perform  independently  of  gravity 

Is  unaffected  by  g  forces 

Can  generate  and  exert  forces  as  needed 

Goes  directly  to  stored  information  for  a  decision 


Has  no  such  limitations 


Has  no  response  latency 
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From  Woodson  (1982) 


ALLOCATION  OF  FUNCTION  (Page  1  of  2) 


Who  Does  What? 

Must  consider  what  people  WILL  do  (characteristic  performance),  not  what  they  can  do  (capability) 

Most  contemporary  decisions  involve  the  LEVEL  and  NATURE  of  semiautomation  that  is  appropriate 
as  opposed  to  traditional  manned  versus  unmanned  systems 

ALLOCATION  OF  FUNCTION  PROCEDURES 
Comparison  of  Human  Capabilities  With  Machine  Alternatives 


HUMAN 

Can  recognize  and  use  information  redundancy 
(pattern)  in  the  real  world  to  simplify  complex 
situations 

Has  high  tolerance  for  ambiguity,  uncertainty, 
and  vagueness 

Can  interpret  an  input  signal  even  when  subject 
to  distraction,  noise,  or  message  gap 

Is  a  selecting  mechanism  and  can  adjust  to  sense 
-.pecific  inputs 

Has  very  low  absolute  thresholds  for  sensing  (e.g. 

.  ision,  audition,  and  touch) 

Has  excellent  long  term  memory  for  related  events 

Can  become  highly  flexible  in  terms  of  task 
performance 

Can  improvise  and  exercise  judgment  on  the  basis 
of  long  term  memory  and  recall 

Can  perform  under  transient  overload; 
performance  degrades  gracefully 

Can  make  inductive  decisions  in  novel  situations: 
can  generalize 

Can  modify  performance  as  a  function  of 
experience;  can  “learn  to  learn” 

Can  override  own  actions,  should  the  need  arise 

Is  reasonably  reliable;  can  add  reliability  to  system 
performance  by  selection  of  alternatives 

Complements  the  machine,  i.e.  can  use  it  in  spue 
of  design  failures,  can  use  it  for  a  different  task, 
or  can  use  it  more  efficiently  than  it  was  designed 
to  be  used 


MACHINE 

Has  limited  perceptual  constancy  and  is  very 
expensive 

Is  highly  limited  by  ambiguity  and  uncertainty  in 
input 

Performs  well  only  in  a  generally  clean,  noise-free 
environment 

Is  a  fixed  sensing  mechanism,  operating  only  on 
that  which  has  been  programmed  for  it 

To  have  the  same  capability,  becomes  extremely 
expensive 

To  have  the  same  capability,  becomes  extremely 
expensive 

Is  relatively  inflexible 

Cannot  do  this;  is  best  at  routine,  repetitive 
functions 

Stops  under  overload;  often  fails  all  at  once 

Has  little  or  no  capability  for  induction  or 
generalization 

Is  not  characterized  by  trial-and-error  behavior 

Can  do  only  what  it  is  built  to  do 

Is  reliable  only  at  the  expense  of  increased 
complexity  and  cost,  and  then  only  for  routine 
functions 

Has  no  such  capability 
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ALLOCATION  OF  FUNCTION  (Page  2  of  2) 


HUMAN 

Complements  the  machine  by  aiding  in  sensing, 
extrapolating,  decisionmaking,  goal  setting, 
monitoring,  and  evaluating 

Can  acquire  and  report  information  incidental  to 
the  primary  mission 

Can  perform  time-contingency  analyses  and 
predict  events  in  unusual  situations 

Is  relatively  inexpensive  for  corresponding 
complexity  and  is  generally  in  good  supply,  but 
must  be  trained 

Is  light  in  weight  and  small  in  size  for  function 
achieved  for  most  situations 

Is  relatively  easy  to  maintain,  demands  a 
minimum  of  “in-task”  extras 


MACHINE 

Has  no  capacity  for  performance  different  from 
what  was  originally  designed 

Cannot  do  this 

Does  very  poorly  at  this 

Is  more  limited  in  terms  of  complexity  and  supply 
by  cost  and  time 

To  have  functional  equivalence  of  the  human, 
requires  more  weight,  power,  and  cooling  facilities 

Maintenance  problems  become  disproportionately 
serious  as  complexity  increases 


From  Woodson  (1982) 


